{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Interacting with Campaigns <a class=\"anchor\" id=\"top\"></a>\n",
    "\n",
    "In this notebook, you will deploy and interact with campaigns in Amazon Personalize.\n",
    "\n",
    "1. [Introduction](#intro)\n",
    "1. [Create campaigns](#create)\n",
    "1. [Interact with campaigns](#interact)\n",
    "1. [Batch recommendations](#batch)\n",
    "1. [Wrap up](#wrapup)\n",
    "\n",
    "## Introduction <a class=\"anchor\" id=\"intro\"></a>\n",
    "[Back to top](#top)\n",
    "\n",
    "At this point, you should have several solutions and at least one solution version for each. Once a solution version is created, it is possible to get recommendations from them, and to get a feel for their overall behavior.\n",
    "\n",
    "This notebook starts off by deploying each of the solution versions from the previous notebook into individual campaigns. Once they are active, there are resources for querying the recommendations, and helper functions to digest the output into something more human-readable. \n",
    "\n",
    "As you with your customer on Amazon Personalize, you can modify the helper functions to fit the structure of their data input files to keep the additional rendering working.\n",
    "\n",
    "To get started, once again, we need to import libraries, load values from previous notebooks, and load the SDK."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/05.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from time import sleep\n",
    "import json\n",
    "from datetime import datetime\n",
    "import uuid\n",
    "import random\n",
    "\n",
    "import boto3\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%store -r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "personalize = boto3.client('personalize')\n",
    "personalize_runtime = boto3.client('personalize-runtime')\n",
    "\n",
    "# Establish a connection to Personalize's event streaming\n",
    "personalize_events = boto3.client(service_name='personalize-events')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interact with campaigns <a class=\"anchor\" id=\"interact\"></a>\n",
    "[Back to top](#top)\n",
    "\n",
    "Now that all campaigns are deployed and active, we can start to get recommendations via an API call. Each of the campaigns is based on a different recipe, which behave in slightly different ways because they serve different use cases. We will cover each campaign in a different order than used in previous notebooks, in order to deal with the possible complexities in ascending order (i.e. simplest first).\n",
    "\n",
    "First, let's create a supporting function to help make sense of the results returned by a Personalize campaign. Personalize returns only an `item_id`. This is great for keeping data compact, but it means you need to query a database or lookup table to get a human-readable result for the notebooks. We will create a helper function to return a human-readable result from the LastFM dataset.\n",
    "\n",
    "Start by loading in the dataset which we can use for our lookup table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>movieId</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Toy Story (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Jumanji (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                      title\n",
       "movieId                                    \n",
       "1                          Toy Story (1995)\n",
       "2                            Jumanji (1995)\n",
       "3                   Grumpier Old Men (1995)\n",
       "4                  Waiting to Exhale (1995)\n",
       "5        Father of the Bride Part II (1995)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a dataframe for the items by reading in the correct source CSV\n",
    "items_df = pd.read_csv(dataset_dir + '/movies.csv', sep=',', usecols=[0,1], encoding='latin-1', dtype={'movieId': \"object\", 'title': \"str\"},index_col=0)\n",
    "\n",
    "# Render some sample data\n",
    "items_df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By defining the ID column as the index column it is trivial to return an artist by just querying the ID."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Terminator 2: Judgment Day (1991)\n"
     ]
    }
   ],
   "source": [
    "movie_id_example = 589\n",
    "title = items_df.loc[movie_id_example]['title']\n",
    "print(title)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That isn't terrible, but it would get messy to repeat this everywhere in our code, so the function below will clean that up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_movie_by_id(movie_id, movie_df=items_df):\n",
    "    \"\"\"\n",
    "    This takes in an artist_id from Personalize so it will be a string,\n",
    "    converts it to an int, and then does a lookup in a default or specified\n",
    "    dataframe.\n",
    "    \n",
    "    A really broad try/except clause was added in case anything goes wrong.\n",
    "    \n",
    "    Feel free to add more debugging or filtering here to improve results if\n",
    "    you hit an error.\n",
    "    \"\"\"\n",
    "    try:\n",
    "        return movie_df.loc[int(movie_id)]['title']\n",
    "    except:\n",
    "        return \"Error obtaining title\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's test a few simple values to check our error catching."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Princess Bride, The (1987)\n"
     ]
    }
   ],
   "source": [
    "# A known good id (The Princess Bride)\n",
    "print(get_movie_by_id(movie_id=\"1197\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error obtaining title\n"
     ]
    }
   ],
   "source": [
    "# A bad type of value\n",
    "print(get_movie_by_id(movie_id=\"987.9393939\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# _The ^^ above ^^ cell should show an error_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error obtaining title\n"
     ]
    }
   ],
   "source": [
    "# Really bad values\n",
    "print(get_movie_by_id(movie_id=\"Steve\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# _The ^^ above ^^ cell should show an error_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Now we have a way of rendering results. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SIMS\n",
    "\n",
    "SIMS requires just an item as input, and it will return items which users interact with in similar ways to their interaction with the input item. In this particular case the item is a movie. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The cells below will handle getting recommendations from SIMS and rendering the results. Let's see what the recommendations are for the first item we looked at earlier in this notebook (Terminator 2: Judgment Day)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "    campaignArn = sims_campaign_arn,\n",
    "    itemId = str(589),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Jurassic Park (1993)\n",
      "Braveheart (1995)\n",
      "Terminator, The (1984)\n",
      "Fugitive, The (1993)\n",
      "Speed (1994)\n",
      "Crimson Tide (1995)\n",
      "GoldenEye (1995)\n",
      "Batman (1989)\n",
      "Clear and Present Danger (1994)\n",
      "True Lies (1994)\n",
      "Mask, The (1994)\n",
      "Die Hard: With a Vengeance (1995)\n",
      "In the Line of Fire (1993)\n",
      "Lion King, The (1994)\n",
      "Ghost (1990)\n",
      "Forrest Gump (1994)\n",
      "Apollo 13 (1995)\n",
      "Cliffhanger (1993)\n",
      "Star Trek: Generations (1994)\n",
      "Firm, The (1993)\n",
      "Die Hard (1988)\n",
      "Seven (a.k.a. Se7en) (1995)\n",
      "Indiana Jones and the Last Crusade (1989)\n",
      "Mission: Impossible (1996)\n",
      "Mrs. Doubtfire (1993)\n"
     ]
    }
   ],
   "source": [
    "item_list = get_recommendations_response['itemList']\n",
    "for item in item_list:\n",
    "    print(get_movie_by_id(movie_id=item['itemId']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Congrats, this is your first list of recommendations! This list is fine, but it would be better to see the recommendations for our sample collection of artists render in a nice dataframe. Again, let's create a helper function to achieve this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Update DF rendering\n",
    "pd.set_option('display.max_rows', 30)\n",
    "\n",
    "def get_new_recommendations_df(recommendations_df, movie_ID):\n",
    "    # Get the movie name\n",
    "    movie_name = get_movie_by_id(movie_ID)\n",
    "    # Get the recommendations\n",
    "    get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "        campaignArn = sims_campaign_arn,\n",
    "        itemId = str(movie_ID),\n",
    "    )\n",
    "    # Build a new dataframe of recommendations\n",
    "    item_list = get_recommendations_response['itemList']\n",
    "    recommendation_list = []\n",
    "    for item in item_list:\n",
    "        movie = get_movie_by_id(item['itemId'])\n",
    "        recommendation_list.append(movie)\n",
    "    new_rec_DF = pd.DataFrame(recommendation_list, columns = [movie_name])\n",
    "    # Add this dataframe to the old one\n",
    "    recommendations_df = pd.concat([recommendations_df, new_rec_DF], axis=1)\n",
    "    return recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's test the helper function with several different movies. Let's sample some data from our dataset to test our SIMS campaign. Grab 5 random movies from our dataframe.\n",
    "\n",
    "Note: We are going to show similar titles, so you may want to re-run the sample until you recognize some of the movies listed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>movieId</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1257</th>\n",
       "      <td>Better Off Dead... (1985)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2275</th>\n",
       "      <td>Six-String Samurai (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6212</th>\n",
       "      <td>Bringing Down the House (2003)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2164</th>\n",
       "      <td>Surf Nazis Must Die (1987)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7841</th>\n",
       "      <td>Children of Dune (2003)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  title\n",
       "movieId                                \n",
       "1257          Better Off Dead... (1985)\n",
       "2275          Six-String Samurai (1998)\n",
       "6212     Bringing Down the House (2003)\n",
       "2164         Surf Nazis Must Die (1987)\n",
       "7841            Children of Dune (2003)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "samples = items_df.sample(5)\n",
    "samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Better Off Dead... (1985)</th>\n",
       "      <th>Six-String Samurai (1998)</th>\n",
       "      <th>Bringing Down the House (2003)</th>\n",
       "      <th>Surf Nazis Must Die (1987)</th>\n",
       "      <th>Children of Dune (2003)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Man on Wire (2008)</td>\n",
       "      <td>Kick-Ass (2010)</td>\n",
       "      <td>Shawshank Redemption, The (1994)</td>\n",
       "      <td>Shawshank Redemption, The (1994)</td>\n",
       "      <td>Shawshank Redemption, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Major League (1989)</td>\n",
       "      <td>Iron Man 2 (2010)</td>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "      <td>Forrest Gump (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Legend (1985)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "      <td>Pulp Fiction (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Anne of Green Gables: The Sequel (a.k.a. Anne ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Silence of the Lambs, The (1991)</td>\n",
       "      <td>Silence of the Lambs, The (1991)</td>\n",
       "      <td>Silence of the Lambs, The (1991)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Jezebel (1938)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Matrix, The (1999)</td>\n",
       "      <td>Matrix, The (1999)</td>\n",
       "      <td>Matrix, The (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Anne of Green Gables (1985)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Braveheart (1995)</td>\n",
       "      <td>Braveheart (1995)</td>\n",
       "      <td>Braveheart (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>She's Out of Control (1989)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Meet the Spartans (2008)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Star Wars: Episode IV - A New Hope (1977)</td>\n",
       "      <td>Star Wars: Episode IV - A New Hope (1977)</td>\n",
       "      <td>Star Wars: Episode IV - A New Hope (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Beautiful (2000)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Jurassic Park (1993)</td>\n",
       "      <td>Jurassic Park (1993)</td>\n",
       "      <td>Jurassic Park (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Bounce (2000)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Terminator 2: Judgment Day (1991)</td>\n",
       "      <td>Terminator 2: Judgment Day (1991)</td>\n",
       "      <td>Terminator 2: Judgment Day (1991)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Head Above Water (1996)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Fugitive, The (1993)</td>\n",
       "      <td>Fugitive, The (1993)</td>\n",
       "      <td>Fugitive, The (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Amazing Grace and Chuck (1987)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Life with Mikey (1993)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Fight Club (1999)</td>\n",
       "      <td>Fight Club (1999)</td>\n",
       "      <td>Fight Club (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>In Crowd, The (2000)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Man from Snowy River, The (1982)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Seven (a.k.a. Se7en) (1995)</td>\n",
       "      <td>Seven (a.k.a. Se7en) (1995)</td>\n",
       "      <td>Seven (a.k.a. Se7en) (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Skulls, The (2000)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Usual Suspects, The (1995)</td>\n",
       "      <td>Usual Suspects, The (1995)</td>\n",
       "      <td>Usual Suspects, The (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Kid, The (2000)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>American Beauty (1999)</td>\n",
       "      <td>American Beauty (1999)</td>\n",
       "      <td>American Beauty (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Left Behind: The Movie (2000)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Raiders of the Lost Ark (Indiana Jones and the...</td>\n",
       "      <td>Raiders of the Lost Ark (Indiana Jones and the...</td>\n",
       "      <td>Raiders of the Lost Ark (Indiana Jones and the...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>My Dog Skip (1999)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Lord of the Rings: The Fellowship of the Ring,...</td>\n",
       "      <td>Lord of the Rings: The Fellowship of the Ring,...</td>\n",
       "      <td>Lord of the Rings: The Fellowship of the Ring,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Brother's Keeper (1992)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Fargo (1996)</td>\n",
       "      <td>Fargo (1996)</td>\n",
       "      <td>Fargo (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Paradise Lost 2: Revelations (2000)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Recount (2008)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Dances with Wolves (1990)</td>\n",
       "      <td>Dances with Wolves (1990)</td>\n",
       "      <td>Dances with Wolves (1990)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>That's Entertainment (1974)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Independence Day (a.k.a. ID4) (1996)</td>\n",
       "      <td>Independence Day (a.k.a. ID4) (1996)</td>\n",
       "      <td>Independence Day (a.k.a. ID4) (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Husbands and Wives (1992)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Lord of the Rings: The Two Towers, The (2002)</td>\n",
       "      <td>Lord of the Rings: The Two Towers, The (2002)</td>\n",
       "      <td>Lord of the Rings: The Two Towers, The (2002)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Romper Stomper (1992)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            Better Off Dead... (1985)  \\\n",
       "0                                  Man on Wire (2008)   \n",
       "1                                 Major League (1989)   \n",
       "2                                       Legend (1985)   \n",
       "3   Anne of Green Gables: The Sequel (a.k.a. Anne ...   \n",
       "4                                      Jezebel (1938)   \n",
       "5                         Anne of Green Gables (1985)   \n",
       "6                         She's Out of Control (1989)   \n",
       "7                            Meet the Spartans (2008)   \n",
       "8                                    Beautiful (2000)   \n",
       "9                                       Bounce (2000)   \n",
       "10                            Head Above Water (1996)   \n",
       "11                     Amazing Grace and Chuck (1987)   \n",
       "12                             Life with Mikey (1993)   \n",
       "13                               In Crowd, The (2000)   \n",
       "14                   Man from Snowy River, The (1982)   \n",
       "15                                 Skulls, The (2000)   \n",
       "16                                    Kid, The (2000)   \n",
       "17                      Left Behind: The Movie (2000)   \n",
       "18                                 My Dog Skip (1999)   \n",
       "19                            Brother's Keeper (1992)   \n",
       "20                Paradise Lost 2: Revelations (2000)   \n",
       "21                                     Recount (2008)   \n",
       "22                        That's Entertainment (1974)   \n",
       "23                          Husbands and Wives (1992)   \n",
       "24                              Romper Stomper (1992)   \n",
       "\n",
       "   Six-String Samurai (1998)  \\\n",
       "0            Kick-Ass (2010)   \n",
       "1          Iron Man 2 (2010)   \n",
       "2                        NaN   \n",
       "3                        NaN   \n",
       "4                        NaN   \n",
       "5                        NaN   \n",
       "6                        NaN   \n",
       "7                        NaN   \n",
       "8                        NaN   \n",
       "9                        NaN   \n",
       "10                       NaN   \n",
       "11                       NaN   \n",
       "12                       NaN   \n",
       "13                       NaN   \n",
       "14                       NaN   \n",
       "15                       NaN   \n",
       "16                       NaN   \n",
       "17                       NaN   \n",
       "18                       NaN   \n",
       "19                       NaN   \n",
       "20                       NaN   \n",
       "21                       NaN   \n",
       "22                       NaN   \n",
       "23                       NaN   \n",
       "24                       NaN   \n",
       "\n",
       "                       Bringing Down the House (2003)  \\\n",
       "0                    Shawshank Redemption, The (1994)   \n",
       "1                                 Forrest Gump (1994)   \n",
       "2                                 Pulp Fiction (1994)   \n",
       "3                    Silence of the Lambs, The (1991)   \n",
       "4                                  Matrix, The (1999)   \n",
       "5                                   Braveheart (1995)   \n",
       "6                             Schindler's List (1993)   \n",
       "7           Star Wars: Episode IV - A New Hope (1977)   \n",
       "8                                Jurassic Park (1993)   \n",
       "9                   Terminator 2: Judgment Day (1991)   \n",
       "10                               Fugitive, The (1993)   \n",
       "11                                   Apollo 13 (1995)   \n",
       "12                                  Fight Club (1999)   \n",
       "13                                   Toy Story (1995)   \n",
       "14                        Seven (a.k.a. Se7en) (1995)   \n",
       "15                         Usual Suspects, The (1995)   \n",
       "16                             American Beauty (1999)   \n",
       "17  Raiders of the Lost Ark (Indiana Jones and the...   \n",
       "18  Lord of the Rings: The Fellowship of the Ring,...   \n",
       "19                                       Fargo (1996)   \n",
       "20  Lord of the Rings: The Return of the King, The...   \n",
       "21                          Dances with Wolves (1990)   \n",
       "22               Independence Day (a.k.a. ID4) (1996)   \n",
       "23      Lord of the Rings: The Two Towers, The (2002)   \n",
       "24                              Godfather, The (1972)   \n",
       "\n",
       "                           Surf Nazis Must Die (1987)  \\\n",
       "0                    Shawshank Redemption, The (1994)   \n",
       "1                                 Forrest Gump (1994)   \n",
       "2                                 Pulp Fiction (1994)   \n",
       "3                    Silence of the Lambs, The (1991)   \n",
       "4                                  Matrix, The (1999)   \n",
       "5                                   Braveheart (1995)   \n",
       "6                             Schindler's List (1993)   \n",
       "7           Star Wars: Episode IV - A New Hope (1977)   \n",
       "8                                Jurassic Park (1993)   \n",
       "9                   Terminator 2: Judgment Day (1991)   \n",
       "10                               Fugitive, The (1993)   \n",
       "11                                   Apollo 13 (1995)   \n",
       "12                                  Fight Club (1999)   \n",
       "13                                   Toy Story (1995)   \n",
       "14                        Seven (a.k.a. Se7en) (1995)   \n",
       "15                         Usual Suspects, The (1995)   \n",
       "16                             American Beauty (1999)   \n",
       "17  Raiders of the Lost Ark (Indiana Jones and the...   \n",
       "18  Lord of the Rings: The Fellowship of the Ring,...   \n",
       "19                                       Fargo (1996)   \n",
       "20  Lord of the Rings: The Return of the King, The...   \n",
       "21                          Dances with Wolves (1990)   \n",
       "22               Independence Day (a.k.a. ID4) (1996)   \n",
       "23      Lord of the Rings: The Two Towers, The (2002)   \n",
       "24                              Godfather, The (1972)   \n",
       "\n",
       "                              Children of Dune (2003)  \n",
       "0                    Shawshank Redemption, The (1994)  \n",
       "1                                 Forrest Gump (1994)  \n",
       "2                                 Pulp Fiction (1994)  \n",
       "3                    Silence of the Lambs, The (1991)  \n",
       "4                                  Matrix, The (1999)  \n",
       "5                                   Braveheart (1995)  \n",
       "6                             Schindler's List (1993)  \n",
       "7           Star Wars: Episode IV - A New Hope (1977)  \n",
       "8                                Jurassic Park (1993)  \n",
       "9                   Terminator 2: Judgment Day (1991)  \n",
       "10                               Fugitive, The (1993)  \n",
       "11                                   Apollo 13 (1995)  \n",
       "12                                  Fight Club (1999)  \n",
       "13                                   Toy Story (1995)  \n",
       "14                        Seven (a.k.a. Se7en) (1995)  \n",
       "15                         Usual Suspects, The (1995)  \n",
       "16                             American Beauty (1999)  \n",
       "17  Raiders of the Lost Ark (Indiana Jones and the...  \n",
       "18  Lord of the Rings: The Fellowship of the Ring,...  \n",
       "19                                       Fargo (1996)  \n",
       "20  Lord of the Rings: The Return of the King, The...  \n",
       "21                          Dances with Wolves (1990)  \n",
       "22               Independence Day (a.k.a. ID4) (1996)  \n",
       "23      Lord of the Rings: The Two Towers, The (2002)  \n",
       "24                              Godfather, The (1972)  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sims_recommendations_df = pd.DataFrame()\n",
    "movies = samples.index.tolist()\n",
    "\n",
    "for movie in movies:\n",
    "    sims_recommendations_df = get_new_recommendations_df(sims_recommendations_df, movie)\n",
    "\n",
    "sims_recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may notice that a lot of the items look the same, hopefully not all of them do (this is more likely with a smaller # of interactions). This shows that the evaluation metrics should not be the only thing you rely on when evaluating your solution version. So when this happens, what can you do to improve the results?\n",
    "\n",
    "This is a good time to think about the hyperparameters of the Personalize recipes. The SIMS recipe has a `popularity_discount_factor` hyperparameter (see [documentation](https://docs.aws.amazon.com/personalize/latest/dg/native-recipe-sims.html)). Leveraging this hyperparameter allows you to control the nuance you see in the results. This parameter and its behavior will be unique to every dataset you encounter, and depends on the goals of the business. You can iterate on the value of this hyperparameter until you are satisfied with the results, or you can start by leveraging Personalize's hyperparameter optimization (HPO) feature. For more information on hyperparameters and HPO tuning, see the [documentation](https://docs.aws.amazon.com/personalize/latest/dg/customizing-solution-config-hpo.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### User Personalization\n",
    "\n",
    "HRNN is one of the more advanced algorithms provided by Amazon Personalize. It supports personalization of the items for a specific user based on their past behavior and can intake real time events in order to alter recommendations for a user without retraining. \n",
    "\n",
    "Since HRNN relies on having a sampling of users, let's load the data we need for that and select 3 random users. Since Movielens does not include user data, we will select 3 random numbers from the range of user id's in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[346, 75, 247]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "if not USE_FULL_MOVIELENS:\n",
    "    users = random.sample(range(1, 600), 3)\n",
    "else:\n",
    "    users = random.sample(range(1, 162000), 3)\n",
    "users"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we render the recommendations for our 3 random users from above. After that, we will explore real-time interactions before moving on to Personalized Ranking.\n",
    "\n",
    "Again, we create a helper function to render the results in a nice dataframe.\n",
    "\n",
    "#### API call results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Update DF rendering\n",
    "pd.set_option('display.max_rows', 30)\n",
    "\n",
    "def get_new_recommendations_df_users(recommendations_df, user_id):\n",
    "    # Get the movie name\n",
    "    #movie_name = get_movie_by_id(artist_ID)\n",
    "    # Get the recommendations\n",
    "    get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "        campaignArn = userpersonalization_campaign_arn,\n",
    "        userId = str(user_id),\n",
    "    )\n",
    "    # Build a new dataframe of recommendations\n",
    "    item_list = get_recommendations_response['itemList']\n",
    "    recommendation_list = []\n",
    "    for item in item_list:\n",
    "        movie = get_movie_by_id(item['itemId'])\n",
    "        recommendation_list.append(movie)\n",
    "    #print(recommendation_list)\n",
    "    new_rec_DF = pd.DataFrame(recommendation_list, columns = [user_id])\n",
    "    # Add this dataframe to the old one\n",
    "    recommendations_df = pd.concat([recommendations_df, new_rec_DF], axis=1)\n",
    "    return recommendations_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>346</th>\n",
       "      <th>75</th>\n",
       "      <th>247</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Blade Runner (1982)</td>\n",
       "      <td>Star Wars: Episode V - The Empire Strikes Back...</td>\n",
       "      <td>Sound of Music, The (1965)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Terminator, The (1984)</td>\n",
       "      <td>Raiders of the Lost Ark (Indiana Jones and the...</td>\n",
       "      <td>Singin' in the Rain (1952)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Rear Window (1954)</td>\n",
       "      <td>Monty Python and the Holy Grail (1975)</td>\n",
       "      <td>Lady and the Tramp (1955)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Ghostbusters (a.k.a. Ghost Busters) (1984)</td>\n",
       "      <td>Rocky (1976)</td>\n",
       "      <td>Cinderella (1950)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Road Warrior, The (Mad Max 2) (1981)</td>\n",
       "      <td>Wizard of Oz, The (1939)</td>\n",
       "      <td>Little Mermaid, The (1989)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Taxi Driver (1976)</td>\n",
       "      <td>Godfather: Part II, The (1974)</td>\n",
       "      <td>My Fair Lady (1964)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>E.T. the Extra-Terrestrial (1982)</td>\n",
       "      <td>Indiana Jones and the Last Crusade (1989)</td>\n",
       "      <td>It's a Wonderful Life (1946)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Professional, The (Le professionnel) (1981)</td>\n",
       "      <td>Back to the Future (1985)</td>\n",
       "      <td>King and I, The (1956)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Star Wars: Episode V - The Empire Strikes Back...</td>\n",
       "      <td>Princess Bride, The (1987)</td>\n",
       "      <td>Meet Me in St. Louis (1944)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Alien (1979)</td>\n",
       "      <td>Shining, The (1980)</td>\n",
       "      <td>Jungle Book, The (1967)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Matrix, The (1999)</td>\n",
       "      <td>Lord of the Rings: The Return of the King, The...</td>\n",
       "      <td>Doctor Dolittle (1967)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Thing, The (1982)</td>\n",
       "      <td>Matrix, The (1999)</td>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Twelve Monkeys (a.k.a. 12 Monkeys) (1995)</td>\n",
       "      <td>Casablanca (1942)</td>\n",
       "      <td>Dumbo (1941)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>WarGames (1983)</td>\n",
       "      <td>Bridge on the River Kwai, The (1957)</td>\n",
       "      <td>Chitty Chitty Bang Bang (1968)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Chinatown (1974)</td>\n",
       "      <td>Alien (1979)</td>\n",
       "      <td>Music Man, The (1962)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Amadeus (1984)</td>\n",
       "      <td>Star Wars: Episode IV - A New Hope (1977)</td>\n",
       "      <td>American in Paris, An (1951)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>All the President's Men (1976)</td>\n",
       "      <td>Godfather, The (1972)</td>\n",
       "      <td>Night at the Opera, A (1935)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Star Wars: Episode IV - A New Hope (1977)</td>\n",
       "      <td>Schindler's List (1993)</td>\n",
       "      <td>Casablanca (1942)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>One Flew Over the Cuckoo's Nest (1975)</td>\n",
       "      <td>American Beauty (1999)</td>\n",
       "      <td>Mulan (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Jurassic Park (1993)</td>\n",
       "      <td>Terminator, The (1984)</td>\n",
       "      <td>North by Northwest (1959)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Star Wars: Episode VI - Return of the Jedi (1983)</td>\n",
       "      <td>Apocalypse Now (1979)</td>\n",
       "      <td>Sword in the Stone, The (1963)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Apocalypse Now (1979)</td>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "      <td>Peter Pan (1953)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>This Is Spinal Tap (1984)</td>\n",
       "      <td>Fight Club (1999)</td>\n",
       "      <td>West Side Story (1961)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Clockwork Orange, A (1971)</td>\n",
       "      <td>Butch Cassidy and the Sundance Kid (1969)</td>\n",
       "      <td>Beauty and the Beast (1991)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Dead Zone, The (1983)</td>\n",
       "      <td>Shawshank Redemption, The (1994)</td>\n",
       "      <td>Alice in Wonderland (1951)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  346  \\\n",
       "0                                 Blade Runner (1982)   \n",
       "1                              Terminator, The (1984)   \n",
       "2                                  Rear Window (1954)   \n",
       "3          Ghostbusters (a.k.a. Ghost Busters) (1984)   \n",
       "4                Road Warrior, The (Mad Max 2) (1981)   \n",
       "5                                  Taxi Driver (1976)   \n",
       "6                   E.T. the Extra-Terrestrial (1982)   \n",
       "7         Professional, The (Le professionnel) (1981)   \n",
       "8   Star Wars: Episode V - The Empire Strikes Back...   \n",
       "9                                        Alien (1979)   \n",
       "10                                 Matrix, The (1999)   \n",
       "11                                  Thing, The (1982)   \n",
       "12          Twelve Monkeys (a.k.a. 12 Monkeys) (1995)   \n",
       "13                                    WarGames (1983)   \n",
       "14                                   Chinatown (1974)   \n",
       "15                                     Amadeus (1984)   \n",
       "16                     All the President's Men (1976)   \n",
       "17          Star Wars: Episode IV - A New Hope (1977)   \n",
       "18             One Flew Over the Cuckoo's Nest (1975)   \n",
       "19                               Jurassic Park (1993)   \n",
       "20  Star Wars: Episode VI - Return of the Jedi (1983)   \n",
       "21                              Apocalypse Now (1979)   \n",
       "22                          This Is Spinal Tap (1984)   \n",
       "23                         Clockwork Orange, A (1971)   \n",
       "24                              Dead Zone, The (1983)   \n",
       "\n",
       "                                                  75   \\\n",
       "0   Star Wars: Episode V - The Empire Strikes Back...   \n",
       "1   Raiders of the Lost Ark (Indiana Jones and the...   \n",
       "2              Monty Python and the Holy Grail (1975)   \n",
       "3                                        Rocky (1976)   \n",
       "4                            Wizard of Oz, The (1939)   \n",
       "5                      Godfather: Part II, The (1974)   \n",
       "6           Indiana Jones and the Last Crusade (1989)   \n",
       "7                           Back to the Future (1985)   \n",
       "8                          Princess Bride, The (1987)   \n",
       "9                                 Shining, The (1980)   \n",
       "10  Lord of the Rings: The Return of the King, The...   \n",
       "11                                 Matrix, The (1999)   \n",
       "12                                  Casablanca (1942)   \n",
       "13               Bridge on the River Kwai, The (1957)   \n",
       "14                                       Alien (1979)   \n",
       "15          Star Wars: Episode IV - A New Hope (1977)   \n",
       "16                              Godfather, The (1972)   \n",
       "17                            Schindler's List (1993)   \n",
       "18                             American Beauty (1999)   \n",
       "19                             Terminator, The (1984)   \n",
       "20                              Apocalypse Now (1979)   \n",
       "21                                   Apollo 13 (1995)   \n",
       "22                                  Fight Club (1999)   \n",
       "23          Butch Cassidy and the Sundance Kid (1969)   \n",
       "24                   Shawshank Redemption, The (1994)   \n",
       "\n",
       "                               247  \n",
       "0       Sound of Music, The (1965)  \n",
       "1       Singin' in the Rain (1952)  \n",
       "2        Lady and the Tramp (1955)  \n",
       "3                Cinderella (1950)  \n",
       "4       Little Mermaid, The (1989)  \n",
       "5              My Fair Lady (1964)  \n",
       "6     It's a Wonderful Life (1946)  \n",
       "7           King and I, The (1956)  \n",
       "8      Meet Me in St. Louis (1944)  \n",
       "9          Jungle Book, The (1967)  \n",
       "10          Doctor Dolittle (1967)  \n",
       "11                  Shrek 2 (2004)  \n",
       "12                    Dumbo (1941)  \n",
       "13  Chitty Chitty Bang Bang (1968)  \n",
       "14           Music Man, The (1962)  \n",
       "15    American in Paris, An (1951)  \n",
       "16    Night at the Opera, A (1935)  \n",
       "17               Casablanca (1942)  \n",
       "18                    Mulan (1998)  \n",
       "19       North by Northwest (1959)  \n",
       "20  Sword in the Stone, The (1963)  \n",
       "21                Peter Pan (1953)  \n",
       "22          West Side Story (1961)  \n",
       "23     Beauty and the Beast (1991)  \n",
       "24      Alice in Wonderland (1951)  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommendations_df_users = pd.DataFrame()\n",
    "#users = users_df.sample(3).index.tolist()\n",
    "\n",
    "for user in users:\n",
    "    recommendations_df_users = get_new_recommendations_df_users(recommendations_df_users, user)\n",
    "\n",
    "recommendations_df_users"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we clearly see that the recommendations for each user are different. If you were to need a cache for these results, you could start by running the API calls through all your users and store the results, or you could use a batch export, which will be covered later in this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets apply item filters to see recommendations for one of these users within a genre\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_new_recommendations_df_by_filter(recommendations_df, user_id, filter_arn):\n",
    "    # Get the movie name\n",
    "    #movie_name = get_movie_by_id(artist_ID)\n",
    "    # Get the recommendations\n",
    "    get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "        campaignArn = userpersonalization_campaign_arn,\n",
    "        userId = str(user_id),\n",
    "        filterArn = filter_arn\n",
    "    )\n",
    "    # Build a new dataframe of recommendations\n",
    "    item_list = get_recommendations_response['itemList']\n",
    "    recommendation_list = []\n",
    "    for item in item_list:\n",
    "        movie = get_movie_by_id(item['itemId'])\n",
    "        recommendation_list.append(movie)\n",
    "    #print(recommendation_list)\n",
    "    filter_name = filter_arn.split('/')[1]\n",
    "    new_rec_DF = pd.DataFrame(recommendation_list, columns = [filter_name])\n",
    "    # Add this dataframe to the old one\n",
    "    recommendations_df = pd.concat([recommendations_df, new_rec_DF], axis=1)\n",
    "    return recommendations_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "247"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the recommendations for movies within a given genre. Within a VOD application you could create Shelves (also known as rails or carosels) easily by using these filters. Depending on the information you have about your items, You could also filter on additional information such as keyword, year/decade etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Fantasy</th>\n",
       "      <th>Western</th>\n",
       "      <th>Comedy</th>\n",
       "      <th>Animation</th>\n",
       "      <th>Action</th>\n",
       "      <th>Thriller</th>\n",
       "      <th>Documentary</th>\n",
       "      <th>1970s</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Cinderella (1950)</td>\n",
       "      <td>Oklahoma! (1955)</td>\n",
       "      <td>Singin' in the Rain (1952)</td>\n",
       "      <td>Lady and the Tramp (1955)</td>\n",
       "      <td>North by Northwest (1959)</td>\n",
       "      <td>North by Northwest (1959)</td>\n",
       "      <td>Monterey Pop (1968)</td>\n",
       "      <td>Robin Hood (1973)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>It's a Wonderful Life (1946)</td>\n",
       "      <td>Seven Brides for Seven Brothers (1954)</td>\n",
       "      <td>Lady and the Tramp (1955)</td>\n",
       "      <td>Cinderella (1950)</td>\n",
       "      <td>Speed (1994)</td>\n",
       "      <td>Vertigo (1958)</td>\n",
       "      <td>Sympathy for the Devil (1968)</td>\n",
       "      <td>Grease (1978)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Chitty Chitty Bang Bang (1968)</td>\n",
       "      <td>Fistful of Dollars, A (Per un pugno di dollari...</td>\n",
       "      <td>Little Mermaid, The (1989)</td>\n",
       "      <td>Little Mermaid, The (1989)</td>\n",
       "      <td>Blues Brothers, The (1980)</td>\n",
       "      <td>To Catch a Thief (1955)</td>\n",
       "      <td>Stop Making Sense (1984)</td>\n",
       "      <td>Muppet Movie, The (1979)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Sword in the Stone, The (1963)</td>\n",
       "      <td>American Tail: Fievel Goes West, An (1991)</td>\n",
       "      <td>My Fair Lady (1964)</td>\n",
       "      <td>Jungle Book, The (1967)</td>\n",
       "      <td>Incredibles, The (2004)</td>\n",
       "      <td>Speed (1994)</td>\n",
       "      <td>In the Realms of the Unreal (2004)</td>\n",
       "      <td>Rocky Horror Picture Show, The (1975)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Peter Pan (1953)</td>\n",
       "      <td>Annie Get Your Gun (1950)</td>\n",
       "      <td>Jungle Book, The (1967)</td>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "      <td>Ben-Hur (1959)</td>\n",
       "      <td>Rear Window (1954)</td>\n",
       "      <td>Hoop Dreams (1994)</td>\n",
       "      <td>Aristocats, The (1970)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Beauty and the Beast (1991)</td>\n",
       "      <td>Good, the Bad and the Ugly, The (Buono, il bru...</td>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "      <td>Dumbo (1941)</td>\n",
       "      <td>Scott Pilgrim vs. the World (2010)</td>\n",
       "      <td>Notorious (1946)</td>\n",
       "      <td>Why Man Creates (1968)</td>\n",
       "      <td>A Cosmic Christmas (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Alice in Wonderland (1951)</td>\n",
       "      <td>Stagecoach (1939)</td>\n",
       "      <td>Chitty Chitty Bang Bang (1968)</td>\n",
       "      <td>Mulan (1998)</td>\n",
       "      <td>Raiders of the Lost Ark (Indiana Jones and the...</td>\n",
       "      <td>Charade (1963)</td>\n",
       "      <td>Pearl Jam Twenty (2011)</td>\n",
       "      <td>Annie Hall (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Brigadoon (1954)</td>\n",
       "      <td>Big Country, The (1958)</td>\n",
       "      <td>Music Man, The (1962)</td>\n",
       "      <td>Sword in the Stone, The (1963)</td>\n",
       "      <td>Great Escape, The (1963)</td>\n",
       "      <td>Rebecca (1940)</td>\n",
       "      <td>Seven Up! (1964)</td>\n",
       "      <td>Pete's Dragon (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Fantasia (1940)</td>\n",
       "      <td>Rio Grande (1950)</td>\n",
       "      <td>Night at the Opera, A (1935)</td>\n",
       "      <td>Peter Pan (1953)</td>\n",
       "      <td>Fistful of Dollars, A (Per un pugno di dollari...</td>\n",
       "      <td>Dracula (Bram Stoker's Dracula) (1992)</td>\n",
       "      <td>Decline of Western Civilization, The (1981)</td>\n",
       "      <td>Charlotte's Web (1973)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Wizard of Oz, The (1939)</td>\n",
       "      <td>Love Me Tender (1956)</td>\n",
       "      <td>Mulan (1998)</td>\n",
       "      <td>Beauty and the Beast (1991)</td>\n",
       "      <td>Zootopia (2016)</td>\n",
       "      <td>Seven (a.k.a. Se7en) (1995)</td>\n",
       "      <td>Madonna: Truth or Dare (1991)</td>\n",
       "      <td>Bedknobs and Broomsticks (1971)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Snow White and the Seven Dwarfs (1937)</td>\n",
       "      <td>Calamity Jane (1953)</td>\n",
       "      <td>Guys and Dolls (1955)</td>\n",
       "      <td>Alice in Wonderland (1951)</td>\n",
       "      <td>Spartacus (1960)</td>\n",
       "      <td>Spellbound (1945)</td>\n",
       "      <td>Secret Policeman's Other Ball, The (1982)</td>\n",
       "      <td>Karlson Returns (1970)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Yellow Submarine (1968)</td>\n",
       "      <td>7 Faces of Dr. Lao (1964)</td>\n",
       "      <td>Hello, Dolly! (1969)</td>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "      <td>Three Musketeers, The (1993)</td>\n",
       "      <td>Talk of the Town, The (1942)</td>\n",
       "      <td>Anvil! The Story of Anvil (2008)</td>\n",
       "      <td>Watership Down (1978)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Mary Poppins (1964)</td>\n",
       "      <td>Giant (1956)</td>\n",
       "      <td>Aladdin (1992)</td>\n",
       "      <td>Fantasia (1940)</td>\n",
       "      <td>Inglourious Basterds (2009)</td>\n",
       "      <td>You Only Live Twice (1967)</td>\n",
       "      <td>Endless Summer, The (1966)</td>\n",
       "      <td>Star Wars: Episode IV - A New Hope (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>How the Grinch Stole Christmas! (1966)</td>\n",
       "      <td>Last of the Mohicans, The (1992)</td>\n",
       "      <td>Robin Hood (1973)</td>\n",
       "      <td>Sleeping Beauty (1959)</td>\n",
       "      <td>Good, the Bad and the Ugly, The (Buono, il bru...</td>\n",
       "      <td>Stage Fright (1950)</td>\n",
       "      <td>Buena Vista Social Club (1999)</td>\n",
       "      <td>Claire's Knee (Genou de Claire, Le) (1970)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Princess and the Frog, The (2009)</td>\n",
       "      <td>Harvey Girls, The (1946)</td>\n",
       "      <td>Yellow Submarine (1968)</td>\n",
       "      <td>Aladdin (1992)</td>\n",
       "      <td>Adventures of Robin Hood, The (1938)</td>\n",
       "      <td>Alphaville (Alphaville, une Ã©trange aventure ...</td>\n",
       "      <td>U2: Rattle and Hum (1988)</td>\n",
       "      <td>Many Adventures of Winnie the Pooh, The (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Rudolph, the Red-Nosed Reindeer (1964)</td>\n",
       "      <td>Paint Your Wagon (1969)</td>\n",
       "      <td>White Christmas (1954)</td>\n",
       "      <td>Robin Hood (1973)</td>\n",
       "      <td>Ice Age: Dawn of the Dinosaurs (2009)</td>\n",
       "      <td>Third Man, The (1949)</td>\n",
       "      <td>Man of Aran (1934)</td>\n",
       "      <td>Manhattan (1979)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Pinocchio (1940)</td>\n",
       "      <td>Go West (1940)</td>\n",
       "      <td>Parent Trap, The (1961)</td>\n",
       "      <td>Snow White and the Seven Dwarfs (1937)</td>\n",
       "      <td>Mask of Zorro, The (1998)</td>\n",
       "      <td>On Her Majesty's Secret Service (1969)</td>\n",
       "      <td>Man Vanishes, A (Ningen Johatsu) (1967)</td>\n",
       "      <td>Phantom Tollbooth, The (1970)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Frozen (2013)</td>\n",
       "      <td>Desperado (1995)</td>\n",
       "      <td>Holiday Inn (1942)</td>\n",
       "      <td>Pocahontas (1995)</td>\n",
       "      <td>Star Wars: Episode IV - A New Hope (1977)</td>\n",
       "      <td>Marnie (1964)</td>\n",
       "      <td>Shine a Light (2008)</td>\n",
       "      <td>Herbie Rides Again (1974)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Shrek (2001)</td>\n",
       "      <td>I Shot Jesse James (1949)</td>\n",
       "      <td>Graduate, The (1967)</td>\n",
       "      <td>Yellow Submarine (1968)</td>\n",
       "      <td>Princess Bride, The (1987)</td>\n",
       "      <td>Armageddon (1998)</td>\n",
       "      <td>Koyaanisqatsi (a.k.a. Koyaanisqatsi: Life Out ...</td>\n",
       "      <td>Scrooge (1970)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Puss in Boots (Nagagutsu o haita neko) (1969)</td>\n",
       "      <td>Broken Arrow (1950)</td>\n",
       "      <td>Mary Poppins (1964)</td>\n",
       "      <td>Winnie the Pooh and the Blustery Day (1968)</td>\n",
       "      <td>Pirates of the Caribbean: The Curse of the Bla...</td>\n",
       "      <td>Goldfinger (1964)</td>\n",
       "      <td>Man with the Movie Camera, The (Chelovek s kin...</td>\n",
       "      <td>Saturday Night Fever (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Absent-Minded Professor, The (1961)</td>\n",
       "      <td>Duel in the Sun (1946)</td>\n",
       "      <td>How the Grinch Stole Christmas! (1966)</td>\n",
       "      <td>Bambi (1942)</td>\n",
       "      <td>Crouching Tiger, Hidden Dragon (Wo hu cang lon...</td>\n",
       "      <td>Aelita: The Queen of Mars (Aelita) (1924)</td>\n",
       "      <td>Mondo Cane (1962)</td>\n",
       "      <td>New York, New York (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Harry Potter and the Half-Blood Prince (2009)</td>\n",
       "      <td>Misfits, The (1961)</td>\n",
       "      <td>Follow the Fleet (1936)</td>\n",
       "      <td>How the Grinch Stole Christmas! (1966)</td>\n",
       "      <td>Mr. &amp; Mrs. Smith (2005)</td>\n",
       "      <td>Diva (1981)</td>\n",
       "      <td>Triumph of the Will (Triumph des Willens) (1934)</td>\n",
       "      <td>Fiddler on the Roof (1971)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Ghost and Mrs. Muir, The (1947)</td>\n",
       "      <td>Little Big Man (1970)</td>\n",
       "      <td>Kiss Me Kate (1953)</td>\n",
       "      <td>Princess and the Frog, The (2009)</td>\n",
       "      <td>The Lego Movie (2014)</td>\n",
       "      <td>Mission: Impossible (1996)</td>\n",
       "      <td>Bill Cosby, Himself (1983)</td>\n",
       "      <td>Zabriskie Point (1970)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Aladdin and the King of Thieves (1996)</td>\n",
       "      <td>Once Upon a Time in the West (C'era una volta ...</td>\n",
       "      <td>Grease (1978)</td>\n",
       "      <td>Rudolph, the Red-Nosed Reindeer (1964)</td>\n",
       "      <td>You Only Live Twice (1967)</td>\n",
       "      <td>Thomas Crown Affair, The (1968)</td>\n",
       "      <td>I Am a Sex Addict (2005)</td>\n",
       "      <td>Love Story (1970)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Scott Pilgrim vs. the World (2010)</td>\n",
       "      <td>Topo, El (1970)</td>\n",
       "      <td>Gay Divorcee, The (1934)</td>\n",
       "      <td>101 Dalmatians (One Hundred and One Dalmatians...</td>\n",
       "      <td>Top Gun (1986)</td>\n",
       "      <td>Secret Life of Walter Mitty, The (1947)</td>\n",
       "      <td>Celluloid Closet, The (1995)</td>\n",
       "      <td>On the Trail of the Bremen Town Musicians (1973)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                          Fantasy  \\\n",
       "0                               Cinderella (1950)   \n",
       "1                    It's a Wonderful Life (1946)   \n",
       "2                  Chitty Chitty Bang Bang (1968)   \n",
       "3                  Sword in the Stone, The (1963)   \n",
       "4                                Peter Pan (1953)   \n",
       "5                     Beauty and the Beast (1991)   \n",
       "6                      Alice in Wonderland (1951)   \n",
       "7                                Brigadoon (1954)   \n",
       "8                                 Fantasia (1940)   \n",
       "9                        Wizard of Oz, The (1939)   \n",
       "10         Snow White and the Seven Dwarfs (1937)   \n",
       "11                        Yellow Submarine (1968)   \n",
       "12                            Mary Poppins (1964)   \n",
       "13         How the Grinch Stole Christmas! (1966)   \n",
       "14              Princess and the Frog, The (2009)   \n",
       "15         Rudolph, the Red-Nosed Reindeer (1964)   \n",
       "16                               Pinocchio (1940)   \n",
       "17                                  Frozen (2013)   \n",
       "18                                   Shrek (2001)   \n",
       "19  Puss in Boots (Nagagutsu o haita neko) (1969)   \n",
       "20            Absent-Minded Professor, The (1961)   \n",
       "21  Harry Potter and the Half-Blood Prince (2009)   \n",
       "22                Ghost and Mrs. Muir, The (1947)   \n",
       "23         Aladdin and the King of Thieves (1996)   \n",
       "24             Scott Pilgrim vs. the World (2010)   \n",
       "\n",
       "                                              Western  \\\n",
       "0                                    Oklahoma! (1955)   \n",
       "1              Seven Brides for Seven Brothers (1954)   \n",
       "2   Fistful of Dollars, A (Per un pugno di dollari...   \n",
       "3          American Tail: Fievel Goes West, An (1991)   \n",
       "4                           Annie Get Your Gun (1950)   \n",
       "5   Good, the Bad and the Ugly, The (Buono, il bru...   \n",
       "6                                   Stagecoach (1939)   \n",
       "7                             Big Country, The (1958)   \n",
       "8                                   Rio Grande (1950)   \n",
       "9                               Love Me Tender (1956)   \n",
       "10                               Calamity Jane (1953)   \n",
       "11                          7 Faces of Dr. Lao (1964)   \n",
       "12                                       Giant (1956)   \n",
       "13                   Last of the Mohicans, The (1992)   \n",
       "14                           Harvey Girls, The (1946)   \n",
       "15                            Paint Your Wagon (1969)   \n",
       "16                                     Go West (1940)   \n",
       "17                                   Desperado (1995)   \n",
       "18                          I Shot Jesse James (1949)   \n",
       "19                                Broken Arrow (1950)   \n",
       "20                             Duel in the Sun (1946)   \n",
       "21                                Misfits, The (1961)   \n",
       "22                              Little Big Man (1970)   \n",
       "23  Once Upon a Time in the West (C'era una volta ...   \n",
       "24                                    Topo, El (1970)   \n",
       "\n",
       "                                    Comedy  \\\n",
       "0               Singin' in the Rain (1952)   \n",
       "1                Lady and the Tramp (1955)   \n",
       "2               Little Mermaid, The (1989)   \n",
       "3                      My Fair Lady (1964)   \n",
       "4                  Jungle Book, The (1967)   \n",
       "5                           Shrek 2 (2004)   \n",
       "6           Chitty Chitty Bang Bang (1968)   \n",
       "7                    Music Man, The (1962)   \n",
       "8             Night at the Opera, A (1935)   \n",
       "9                             Mulan (1998)   \n",
       "10                   Guys and Dolls (1955)   \n",
       "11                    Hello, Dolly! (1969)   \n",
       "12                          Aladdin (1992)   \n",
       "13                       Robin Hood (1973)   \n",
       "14                 Yellow Submarine (1968)   \n",
       "15                  White Christmas (1954)   \n",
       "16                 Parent Trap, The (1961)   \n",
       "17                      Holiday Inn (1942)   \n",
       "18                    Graduate, The (1967)   \n",
       "19                     Mary Poppins (1964)   \n",
       "20  How the Grinch Stole Christmas! (1966)   \n",
       "21                 Follow the Fleet (1936)   \n",
       "22                     Kiss Me Kate (1953)   \n",
       "23                           Grease (1978)   \n",
       "24                Gay Divorcee, The (1934)   \n",
       "\n",
       "                                            Animation  \\\n",
       "0                           Lady and the Tramp (1955)   \n",
       "1                                   Cinderella (1950)   \n",
       "2                          Little Mermaid, The (1989)   \n",
       "3                             Jungle Book, The (1967)   \n",
       "4                                      Shrek 2 (2004)   \n",
       "5                                        Dumbo (1941)   \n",
       "6                                        Mulan (1998)   \n",
       "7                      Sword in the Stone, The (1963)   \n",
       "8                                    Peter Pan (1953)   \n",
       "9                         Beauty and the Beast (1991)   \n",
       "10                         Alice in Wonderland (1951)   \n",
       "11                              Lion King, The (1994)   \n",
       "12                                    Fantasia (1940)   \n",
       "13                             Sleeping Beauty (1959)   \n",
       "14                                     Aladdin (1992)   \n",
       "15                                  Robin Hood (1973)   \n",
       "16             Snow White and the Seven Dwarfs (1937)   \n",
       "17                                  Pocahontas (1995)   \n",
       "18                            Yellow Submarine (1968)   \n",
       "19        Winnie the Pooh and the Blustery Day (1968)   \n",
       "20                                       Bambi (1942)   \n",
       "21             How the Grinch Stole Christmas! (1966)   \n",
       "22                  Princess and the Frog, The (2009)   \n",
       "23             Rudolph, the Red-Nosed Reindeer (1964)   \n",
       "24  101 Dalmatians (One Hundred and One Dalmatians...   \n",
       "\n",
       "                                               Action  \\\n",
       "0                           North by Northwest (1959)   \n",
       "1                                        Speed (1994)   \n",
       "2                          Blues Brothers, The (1980)   \n",
       "3                             Incredibles, The (2004)   \n",
       "4                                      Ben-Hur (1959)   \n",
       "5                  Scott Pilgrim vs. the World (2010)   \n",
       "6   Raiders of the Lost Ark (Indiana Jones and the...   \n",
       "7                            Great Escape, The (1963)   \n",
       "8   Fistful of Dollars, A (Per un pugno di dollari...   \n",
       "9                                     Zootopia (2016)   \n",
       "10                                   Spartacus (1960)   \n",
       "11                       Three Musketeers, The (1993)   \n",
       "12                        Inglourious Basterds (2009)   \n",
       "13  Good, the Bad and the Ugly, The (Buono, il bru...   \n",
       "14               Adventures of Robin Hood, The (1938)   \n",
       "15              Ice Age: Dawn of the Dinosaurs (2009)   \n",
       "16                          Mask of Zorro, The (1998)   \n",
       "17          Star Wars: Episode IV - A New Hope (1977)   \n",
       "18                         Princess Bride, The (1987)   \n",
       "19  Pirates of the Caribbean: The Curse of the Bla...   \n",
       "20  Crouching Tiger, Hidden Dragon (Wo hu cang lon...   \n",
       "21                            Mr. & Mrs. Smith (2005)   \n",
       "22                              The Lego Movie (2014)   \n",
       "23                         You Only Live Twice (1967)   \n",
       "24                                     Top Gun (1986)   \n",
       "\n",
       "                                             Thriller  \\\n",
       "0                           North by Northwest (1959)   \n",
       "1                                      Vertigo (1958)   \n",
       "2                             To Catch a Thief (1955)   \n",
       "3                                        Speed (1994)   \n",
       "4                                  Rear Window (1954)   \n",
       "5                                    Notorious (1946)   \n",
       "6                                      Charade (1963)   \n",
       "7                                      Rebecca (1940)   \n",
       "8              Dracula (Bram Stoker's Dracula) (1992)   \n",
       "9                         Seven (a.k.a. Se7en) (1995)   \n",
       "10                                  Spellbound (1945)   \n",
       "11                       Talk of the Town, The (1942)   \n",
       "12                         You Only Live Twice (1967)   \n",
       "13                                Stage Fright (1950)   \n",
       "14  Alphaville (Alphaville, une Ã©trange aventure ...   \n",
       "15                              Third Man, The (1949)   \n",
       "16             On Her Majesty's Secret Service (1969)   \n",
       "17                                      Marnie (1964)   \n",
       "18                                  Armageddon (1998)   \n",
       "19                                  Goldfinger (1964)   \n",
       "20          Aelita: The Queen of Mars (Aelita) (1924)   \n",
       "21                                        Diva (1981)   \n",
       "22                         Mission: Impossible (1996)   \n",
       "23                    Thomas Crown Affair, The (1968)   \n",
       "24            Secret Life of Walter Mitty, The (1947)   \n",
       "\n",
       "                                          Documentary  \\\n",
       "0                                 Monterey Pop (1968)   \n",
       "1                       Sympathy for the Devil (1968)   \n",
       "2                            Stop Making Sense (1984)   \n",
       "3                  In the Realms of the Unreal (2004)   \n",
       "4                                  Hoop Dreams (1994)   \n",
       "5                              Why Man Creates (1968)   \n",
       "6                             Pearl Jam Twenty (2011)   \n",
       "7                                    Seven Up! (1964)   \n",
       "8         Decline of Western Civilization, The (1981)   \n",
       "9                       Madonna: Truth or Dare (1991)   \n",
       "10          Secret Policeman's Other Ball, The (1982)   \n",
       "11                   Anvil! The Story of Anvil (2008)   \n",
       "12                         Endless Summer, The (1966)   \n",
       "13                     Buena Vista Social Club (1999)   \n",
       "14                          U2: Rattle and Hum (1988)   \n",
       "15                                 Man of Aran (1934)   \n",
       "16            Man Vanishes, A (Ningen Johatsu) (1967)   \n",
       "17                               Shine a Light (2008)   \n",
       "18  Koyaanisqatsi (a.k.a. Koyaanisqatsi: Life Out ...   \n",
       "19  Man with the Movie Camera, The (Chelovek s kin...   \n",
       "20                                  Mondo Cane (1962)   \n",
       "21   Triumph of the Will (Triumph des Willens) (1934)   \n",
       "22                         Bill Cosby, Himself (1983)   \n",
       "23                           I Am a Sex Addict (2005)   \n",
       "24                       Celluloid Closet, The (1995)   \n",
       "\n",
       "                                               1970s  \n",
       "0                                  Robin Hood (1973)  \n",
       "1                                      Grease (1978)  \n",
       "2                           Muppet Movie, The (1979)  \n",
       "3              Rocky Horror Picture Show, The (1975)  \n",
       "4                             Aristocats, The (1970)  \n",
       "5                          A Cosmic Christmas (1977)  \n",
       "6                                  Annie Hall (1977)  \n",
       "7                               Pete's Dragon (1977)  \n",
       "8                             Charlotte's Web (1973)  \n",
       "9                    Bedknobs and Broomsticks (1971)  \n",
       "10                            Karlson Returns (1970)  \n",
       "11                             Watership Down (1978)  \n",
       "12         Star Wars: Episode IV - A New Hope (1977)  \n",
       "13        Claire's Knee (Genou de Claire, Le) (1970)  \n",
       "14    Many Adventures of Winnie the Pooh, The (1977)  \n",
       "15                                  Manhattan (1979)  \n",
       "16                     Phantom Tollbooth, The (1970)  \n",
       "17                         Herbie Rides Again (1974)  \n",
       "18                                    Scrooge (1970)  \n",
       "19                       Saturday Night Fever (1977)  \n",
       "20                         New York, New York (1977)  \n",
       "21                        Fiddler on the Roof (1971)  \n",
       "22                            Zabriskie Point (1970)  \n",
       "23                                 Love Story (1970)  \n",
       "24  On the Trail of the Bremen Town Musicians (1973)  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommendations_df_shelves = pd.DataFrame()\n",
    "for filter_arn in meta_filter_arns:\n",
    "    recommendations_df_shelves = get_new_recommendations_df_by_filter(recommendations_df_shelves, user, filter_arn)\n",
    "for filter_arn in decade_filter_arns:\n",
    "    recommendations_df_shelves = get_new_recommendations_df_by_filter(recommendations_df_shelves, user, filter_arn)\n",
    "\n",
    "recommendations_df_shelves"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next topic is real-time events. Personalize has the ability to listen to events from your application in order to update the recommendations shown to the user. This is especially useful in media workloads, like video-on-demand, where a customer's intent may differ based on if they are watching with their children or on their own.\n",
    "\n",
    "Additionally the events that are recorded via this system are stored until a delete call from you is issued, and they are used as historical data alongside the other interaction data you provided when you train your next models.\n",
    "\n",
    "#### Real time events\n",
    "\n",
    "Start by creating an event tracker that is attached to the campaign."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arn:aws:personalize:us-east-1:835319576252:event-tracker/1e2b508b\n",
      "b5b715b6-6d9d-4166-a64b-e320f968c84d\n"
     ]
    }
   ],
   "source": [
    "response = personalize.create_event_tracker(\n",
    "    name='MovieTracker',\n",
    "    datasetGroupArn=dataset_group_arn\n",
    ")\n",
    "print(response['eventTrackerArn'])\n",
    "print(response['trackingId'])\n",
    "TRACKING_ID = response['trackingId']\n",
    "event_tracker_arn = response['eventTrackerArn']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will create some code that simulates a user interacting with a particular item. After running this code, you will get recommendations that differ from the results above.\n",
    "\n",
    "We start by creating some methods for the simulation of real time events."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "session_dict = {}\n",
    "\n",
    "def send_movie_click(USER_ID, ITEM_ID, EVENT_TYPE):\n",
    "    \"\"\"\n",
    "    Simulates a click as an envent\n",
    "    to send an event to Amazon Personalize's Event Tracker\n",
    "    \"\"\"\n",
    "    # Configure Session\n",
    "    try:\n",
    "        session_ID = session_dict[str(USER_ID)]\n",
    "    except:\n",
    "        session_dict[str(USER_ID)] = str(uuid.uuid1())\n",
    "        session_ID = session_dict[str(USER_ID)]\n",
    "        \n",
    "    # Configure Properties:\n",
    "    event = {\n",
    "    \"itemId\": str(ITEM_ID),\n",
    "    }\n",
    "    event_json = json.dumps(event)\n",
    "        \n",
    "    # Make Call\n",
    "    \n",
    "    personalize_events.put_events(\n",
    "    trackingId = TRACKING_ID,\n",
    "    userId= str(USER_ID),\n",
    "    sessionId = session_ID,\n",
    "    eventList = [{\n",
    "        'sentAt': int(time.time()),\n",
    "        'eventType': str(EVENT_TYPE),\n",
    "        'properties': event_json\n",
    "        }]\n",
    "    )\n",
    "\n",
    "def get_new_recommendations_df_users_real_time(recommendations_df, user_id, item_id, event_type):\n",
    "    # Get the artist name (header of column)\n",
    "    movie_name = get_movie_by_id(item_id)\n",
    "    # Interact with different movies\n",
    "    print('sending event ' + event_type + ' for ' + get_movie_by_id(item_id))\n",
    "    send_movie_click(USER_ID=user_id, ITEM_ID=item_id, EVENT_TYPE=event_type)\n",
    "    # Get the recommendations (note you should have a base recommendation DF created before)\n",
    "    get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "        campaignArn = userpersonalization_campaign_arn,\n",
    "        userId = str(user_id),\n",
    "    )\n",
    "    # Build a new dataframe of recommendations\n",
    "    item_list = get_recommendations_response['itemList']\n",
    "    recommendation_list = []\n",
    "    for item in item_list:\n",
    "        artist = get_movie_by_id(item['itemId'])\n",
    "        recommendation_list.append(artist)\n",
    "    new_rec_DF = pd.DataFrame(recommendation_list, columns = [movie_name])\n",
    "    # Add this dataframe to the old one\n",
    "    #recommendations_df = recommendations_df.join(new_rec_DF)\n",
    "    recommendations_df = pd.concat([recommendations_df, new_rec_DF], axis=1)\n",
    "    return recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, we haven't generated any real-time events yet; we have only set up the code. To compare the recommendations before and after the real-time events, let's pick one user and generate the original recommendations for them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>247</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Sound of Music, The (1965)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Singin' in the Rain (1952)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lady and the Tramp (1955)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Cinderella (1950)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Little Mermaid, The (1989)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>My Fair Lady (1964)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>It's a Wonderful Life (1946)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>King and I, The (1956)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Meet Me in St. Louis (1944)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Jungle Book, The (1967)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Doctor Dolittle (1967)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Dumbo (1941)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Chitty Chitty Bang Bang (1968)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Music Man, The (1962)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>American in Paris, An (1951)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Night at the Opera, A (1935)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Casablanca (1942)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Mulan (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>North by Northwest (1959)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Sword in the Stone, The (1963)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Peter Pan (1953)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>West Side Story (1961)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Beauty and the Beast (1991)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Alice in Wonderland (1951)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                               247\n",
       "0       Sound of Music, The (1965)\n",
       "1       Singin' in the Rain (1952)\n",
       "2        Lady and the Tramp (1955)\n",
       "3                Cinderella (1950)\n",
       "4       Little Mermaid, The (1989)\n",
       "5              My Fair Lady (1964)\n",
       "6     It's a Wonderful Life (1946)\n",
       "7           King and I, The (1956)\n",
       "8      Meet Me in St. Louis (1944)\n",
       "9          Jungle Book, The (1967)\n",
       "10          Doctor Dolittle (1967)\n",
       "11                  Shrek 2 (2004)\n",
       "12                    Dumbo (1941)\n",
       "13  Chitty Chitty Bang Bang (1968)\n",
       "14           Music Man, The (1962)\n",
       "15    American in Paris, An (1951)\n",
       "16    Night at the Opera, A (1935)\n",
       "17               Casablanca (1942)\n",
       "18                    Mulan (1998)\n",
       "19       North by Northwest (1959)\n",
       "20  Sword in the Stone, The (1963)\n",
       "21                Peter Pan (1953)\n",
       "22          West Side Story (1961)\n",
       "23     Beauty and the Beast (1991)\n",
       "24      Alice in Wonderland (1951)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# First pick a user\n",
    "user_id = user\n",
    "\n",
    "# Get recommendations for the user\n",
    "get_recommendations_response = personalize_runtime.get_recommendations(\n",
    "        campaignArn = userpersonalization_campaign_arn,\n",
    "        userId = str(user_id),\n",
    "    )\n",
    "\n",
    "# Build a new dataframe for the recommendations\n",
    "item_list = get_recommendations_response['itemList']\n",
    "recommendation_list = []\n",
    "for item in item_list:\n",
    "    artist = get_movie_by_id(item['itemId'])\n",
    "    recommendation_list.append(artist)\n",
    "user_recommendations_df = pd.DataFrame(recommendation_list, columns = [user_id])\n",
    "user_recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok, so now we have a list of recommendations for this user before we have applied any real-time events. Now let's pick 3 random artists which we will simulate our user interacting with, and then see how this changes the recommendations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Next generate 3 random movies\n",
    "movies = items_df.sample(3).index.tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sending event click for Super Size Me (2004)\n",
      "sending event click for Transporter 3 (2008)\n",
      "sending event click for Stoning of Soraya M., The (2008)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>247</th>\n",
       "      <th>Super Size Me (2004)</th>\n",
       "      <th>Transporter 3 (2008)</th>\n",
       "      <th>Stoning of Soraya M., The (2008)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Sound of Music, The (1965)</td>\n",
       "      <td>Sound of Music, The (1965)</td>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Singin' in the Rain (1952)</td>\n",
       "      <td>Singin' in the Rain (1952)</td>\n",
       "      <td>40-Year-Old Virgin, The (2005)</td>\n",
       "      <td>Casino Royale (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lady and the Tramp (1955)</td>\n",
       "      <td>Lady and the Tramp (1955)</td>\n",
       "      <td>Lost in Translation (2003)</td>\n",
       "      <td>Little Miss Sunshine (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Cinderella (1950)</td>\n",
       "      <td>Cinderella (1950)</td>\n",
       "      <td>Eternal Sunshine of the Spotless Mind (2004)</td>\n",
       "      <td>Avatar (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Little Mermaid, The (1989)</td>\n",
       "      <td>Little Mermaid, The (1989)</td>\n",
       "      <td>Howl's Moving Castle (Hauru no ugoku shiro) (2...</td>\n",
       "      <td>Mr. &amp; Mrs. Smith (2005)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>My Fair Lady (1964)</td>\n",
       "      <td>My Fair Lady (1964)</td>\n",
       "      <td>Bruce Almighty (2003)</td>\n",
       "      <td>Pirates of the Caribbean: Dead Man's Chest (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>It's a Wonderful Life (1946)</td>\n",
       "      <td>It's a Wonderful Life (1946)</td>\n",
       "      <td>Finding Nemo (2003)</td>\n",
       "      <td>Mulan (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>King and I, The (1956)</td>\n",
       "      <td>King and I, The (1956)</td>\n",
       "      <td>Terminal, The (2004)</td>\n",
       "      <td>Stardust (2007)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Meet Me in St. Louis (1944)</td>\n",
       "      <td>Meet Me in St. Louis (1944)</td>\n",
       "      <td>Love Actually (2003)</td>\n",
       "      <td>Harry Potter and the Order of the Phoenix (2007)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Jungle Book, The (1967)</td>\n",
       "      <td>Jungle Book, The (1967)</td>\n",
       "      <td>Good bye, Lenin! (2003)</td>\n",
       "      <td>Star Trek (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Doctor Dolittle (1967)</td>\n",
       "      <td>Doctor Dolittle (1967)</td>\n",
       "      <td>50 First Dates (2004)</td>\n",
       "      <td>Iron Man (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "      <td>Little Mermaid, The (1989)</td>\n",
       "      <td>Jungle Book, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Dumbo (1941)</td>\n",
       "      <td>Dumbo (1941)</td>\n",
       "      <td>Mrs. Doubtfire (1993)</td>\n",
       "      <td>Robots (2005)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Chitty Chitty Bang Bang (1968)</td>\n",
       "      <td>Chitty Chitty Bang Bang (1968)</td>\n",
       "      <td>Pirates of the Caribbean: The Curse of the Bla...</td>\n",
       "      <td>WALLÂ·E (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Music Man, The (1962)</td>\n",
       "      <td>Music Man, The (1962)</td>\n",
       "      <td>Something's Gotta Give (2003)</td>\n",
       "      <td>Meet the Robinsons (2007)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>American in Paris, An (1951)</td>\n",
       "      <td>American in Paris, An (1951)</td>\n",
       "      <td>Big Fish (2003)</td>\n",
       "      <td>Pirates of the Caribbean: At World's End (2007)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Night at the Opera, A (1935)</td>\n",
       "      <td>Night at the Opera, A (1935)</td>\n",
       "      <td>Robots (2005)</td>\n",
       "      <td>Raiders of the Lost Ark (Indiana Jones and the...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Casablanca (1942)</td>\n",
       "      <td>Casablanca (1942)</td>\n",
       "      <td>Walk the Line (2005)</td>\n",
       "      <td>Bourne Ultimatum, The (2007)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Mulan (1998)</td>\n",
       "      <td>Mulan (1998)</td>\n",
       "      <td>Wallace &amp; Gromit in The Curse of the Were-Rabb...</td>\n",
       "      <td>Mutiny on the Bounty (1962)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>North by Northwest (1959)</td>\n",
       "      <td>North by Northwest (1959)</td>\n",
       "      <td>Corpse Bride (2005)</td>\n",
       "      <td>40-Year-Old Virgin, The (2005)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Sword in the Stone, The (1963)</td>\n",
       "      <td>Sword in the Stone, The (1963)</td>\n",
       "      <td>Pride &amp; Prejudice (2005)</td>\n",
       "      <td>Incredibles, The (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Peter Pan (1953)</td>\n",
       "      <td>Peter Pan (1953)</td>\n",
       "      <td>When Harry Met Sally... (1989)</td>\n",
       "      <td>Spider-Man 3 (2007)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>West Side Story (1961)</td>\n",
       "      <td>West Side Story (1961)</td>\n",
       "      <td>Super Size Me (2004)</td>\n",
       "      <td>Dead Man Walking (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Beauty and the Beast (1991)</td>\n",
       "      <td>Beauty and the Beast (1991)</td>\n",
       "      <td>Wedding Crashers (2005)</td>\n",
       "      <td>Ice Age: Dawn of the Dinosaurs (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Alice in Wonderland (1951)</td>\n",
       "      <td>Alice in Wonderland (1951)</td>\n",
       "      <td>School of Rock (2003)</td>\n",
       "      <td>Kung Fu Panda (2008)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                               247            Super Size Me (2004)  \\\n",
       "0       Sound of Music, The (1965)      Sound of Music, The (1965)   \n",
       "1       Singin' in the Rain (1952)      Singin' in the Rain (1952)   \n",
       "2        Lady and the Tramp (1955)       Lady and the Tramp (1955)   \n",
       "3                Cinderella (1950)               Cinderella (1950)   \n",
       "4       Little Mermaid, The (1989)      Little Mermaid, The (1989)   \n",
       "5              My Fair Lady (1964)             My Fair Lady (1964)   \n",
       "6     It's a Wonderful Life (1946)    It's a Wonderful Life (1946)   \n",
       "7           King and I, The (1956)          King and I, The (1956)   \n",
       "8      Meet Me in St. Louis (1944)     Meet Me in St. Louis (1944)   \n",
       "9          Jungle Book, The (1967)         Jungle Book, The (1967)   \n",
       "10          Doctor Dolittle (1967)          Doctor Dolittle (1967)   \n",
       "11                  Shrek 2 (2004)                  Shrek 2 (2004)   \n",
       "12                    Dumbo (1941)                    Dumbo (1941)   \n",
       "13  Chitty Chitty Bang Bang (1968)  Chitty Chitty Bang Bang (1968)   \n",
       "14           Music Man, The (1962)           Music Man, The (1962)   \n",
       "15    American in Paris, An (1951)    American in Paris, An (1951)   \n",
       "16    Night at the Opera, A (1935)    Night at the Opera, A (1935)   \n",
       "17               Casablanca (1942)               Casablanca (1942)   \n",
       "18                    Mulan (1998)                    Mulan (1998)   \n",
       "19       North by Northwest (1959)       North by Northwest (1959)   \n",
       "20  Sword in the Stone, The (1963)  Sword in the Stone, The (1963)   \n",
       "21                Peter Pan (1953)                Peter Pan (1953)   \n",
       "22          West Side Story (1961)          West Side Story (1961)   \n",
       "23     Beauty and the Beast (1991)     Beauty and the Beast (1991)   \n",
       "24      Alice in Wonderland (1951)      Alice in Wonderland (1951)   \n",
       "\n",
       "                                 Transporter 3 (2008)  \\\n",
       "0                                      Shrek 2 (2004)   \n",
       "1                      40-Year-Old Virgin, The (2005)   \n",
       "2                          Lost in Translation (2003)   \n",
       "3        Eternal Sunshine of the Spotless Mind (2004)   \n",
       "4   Howl's Moving Castle (Hauru no ugoku shiro) (2...   \n",
       "5                               Bruce Almighty (2003)   \n",
       "6                                 Finding Nemo (2003)   \n",
       "7                                Terminal, The (2004)   \n",
       "8                                Love Actually (2003)   \n",
       "9                             Good bye, Lenin! (2003)   \n",
       "10                              50 First Dates (2004)   \n",
       "11                         Little Mermaid, The (1989)   \n",
       "12                              Mrs. Doubtfire (1993)   \n",
       "13  Pirates of the Caribbean: The Curse of the Bla...   \n",
       "14                      Something's Gotta Give (2003)   \n",
       "15                                    Big Fish (2003)   \n",
       "16                                      Robots (2005)   \n",
       "17                               Walk the Line (2005)   \n",
       "18  Wallace & Gromit in The Curse of the Were-Rabb...   \n",
       "19                                Corpse Bride (2005)   \n",
       "20                           Pride & Prejudice (2005)   \n",
       "21                     When Harry Met Sally... (1989)   \n",
       "22                               Super Size Me (2004)   \n",
       "23                            Wedding Crashers (2005)   \n",
       "24                              School of Rock (2003)   \n",
       "\n",
       "                     Stoning of Soraya M., The (2008)  \n",
       "0                                      Shrek 2 (2004)  \n",
       "1                                Casino Royale (2006)  \n",
       "2                         Little Miss Sunshine (2006)  \n",
       "3                                       Avatar (2009)  \n",
       "4                             Mr. & Mrs. Smith (2005)  \n",
       "5   Pirates of the Caribbean: Dead Man's Chest (2006)  \n",
       "6                                        Mulan (1998)  \n",
       "7                                     Stardust (2007)  \n",
       "8    Harry Potter and the Order of the Phoenix (2007)  \n",
       "9                                    Star Trek (2009)  \n",
       "10                                    Iron Man (2008)  \n",
       "11                            Jungle Book, The (1994)  \n",
       "12                                      Robots (2005)  \n",
       "13                                     WALLÂ·E (2008)  \n",
       "14                          Meet the Robinsons (2007)  \n",
       "15    Pirates of the Caribbean: At World's End (2007)  \n",
       "16  Raiders of the Lost Ark (Indiana Jones and the...  \n",
       "17                       Bourne Ultimatum, The (2007)  \n",
       "18                        Mutiny on the Bounty (1962)  \n",
       "19                     40-Year-Old Virgin, The (2005)  \n",
       "20                            Incredibles, The (2004)  \n",
       "21                                Spider-Man 3 (2007)  \n",
       "22                            Dead Man Walking (1995)  \n",
       "23              Ice Age: Dawn of the Dinosaurs (2009)  \n",
       "24                               Kung Fu Panda (2008)  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Note this will take about 15 seconds to complete due to the sleeps\n",
    "for movie in movies:\n",
    "    time.sleep(5)\n",
    "    user_recommendations_df = get_new_recommendations_df_users_real_time(user_recommendations_df, user_id, movie,'click')\n",
    "user_recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the cell above, the first column after the index is the user's default recommendations from HRNN, and each column after that has a header of the artist that they interacted with via a real time event, and the recommendations after this event occurred. \n",
    "\n",
    "The behavior may not shift very much; this is due to the relatively limited nature of this dataset and effect of a few random clicks. If you wanted to better understand this, try simulating clicking more movies, and you should see a more pronounced impact."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets look at the event filters, which allow you to filter items based on the interaction data. For this dataset, it could be click or watch based on the data we imported, but could be based on whatever interaction schema you design (click, rate, like, watch, purchase etc.) For VOD shelves you could move a title from \"Top picks for you\" to a \"Watch again\", the watch again recommendations will be based on the users current interactions, but only recommend titles that have already been watched.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>watched</th>\n",
       "      <th>unwatched</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Harry Potter and the Order of the Phoenix (2007)</td>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Star Trek (2009)</td>\n",
       "      <td>Casino Royale (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Bourne Ultimatum, The (2007)</td>\n",
       "      <td>Little Miss Sunshine (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Incredibles, The (2004)</td>\n",
       "      <td>Avatar (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Mr. &amp; Mrs. Smith (2005)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Pirates of the Caribbean: Dead Man's Chest (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Mulan (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Stardust (2007)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Iron Man (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Jungle Book, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Robots (2005)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>NaN</td>\n",
       "      <td>WALLÂ·E (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Meet the Robinsons (2007)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Pirates of the Caribbean: At World's End (2007)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Raiders of the Lost Ark (Indiana Jones and the...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>NaN</td>\n",
       "      <td>40-Year-Old Virgin, The (2005)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Mutiny on the Bounty (1962)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Spider-Man 3 (2007)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Dead Man Walking (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Ice Age: Dawn of the Dinosaurs (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Kung Fu Panda (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Thelma &amp; Louise (1991)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Indiana Jones and the Kingdom of the Crystal S...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Brothers Bloom, The (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Blood Diamond (2006)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             watched  \\\n",
       "0   Harry Potter and the Order of the Phoenix (2007)   \n",
       "1                                   Star Trek (2009)   \n",
       "2                       Bourne Ultimatum, The (2007)   \n",
       "3                            Incredibles, The (2004)   \n",
       "4                                                NaN   \n",
       "5                                                NaN   \n",
       "6                                                NaN   \n",
       "7                                                NaN   \n",
       "8                                                NaN   \n",
       "9                                                NaN   \n",
       "10                                               NaN   \n",
       "11                                               NaN   \n",
       "12                                               NaN   \n",
       "13                                               NaN   \n",
       "14                                               NaN   \n",
       "15                                               NaN   \n",
       "16                                               NaN   \n",
       "17                                               NaN   \n",
       "18                                               NaN   \n",
       "19                                               NaN   \n",
       "20                                               NaN   \n",
       "21                                               NaN   \n",
       "22                                               NaN   \n",
       "23                                               NaN   \n",
       "24                                               NaN   \n",
       "\n",
       "                                            unwatched  \n",
       "0                                      Shrek 2 (2004)  \n",
       "1                                Casino Royale (2006)  \n",
       "2                         Little Miss Sunshine (2006)  \n",
       "3                                       Avatar (2009)  \n",
       "4                             Mr. & Mrs. Smith (2005)  \n",
       "5   Pirates of the Caribbean: Dead Man's Chest (2006)  \n",
       "6                                        Mulan (1998)  \n",
       "7                                     Stardust (2007)  \n",
       "8                                     Iron Man (2008)  \n",
       "9                             Jungle Book, The (1994)  \n",
       "10                                      Robots (2005)  \n",
       "11                                     WALLÂ·E (2008)  \n",
       "12                          Meet the Robinsons (2007)  \n",
       "13    Pirates of the Caribbean: At World's End (2007)  \n",
       "14  Raiders of the Lost Ark (Indiana Jones and the...  \n",
       "15                     40-Year-Old Virgin, The (2005)  \n",
       "16                        Mutiny on the Bounty (1962)  \n",
       "17                                Spider-Man 3 (2007)  \n",
       "18                            Dead Man Walking (1995)  \n",
       "19              Ice Age: Dawn of the Dinosaurs (2009)  \n",
       "20                               Kung Fu Panda (2008)  \n",
       "21                             Thelma & Louise (1991)  \n",
       "22  Indiana Jones and the Kingdom of the Crystal S...  \n",
       "23                         Brothers Bloom, The (2008)  \n",
       "24                               Blood Diamond (2006)  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommendations_df_events = pd.DataFrame()\n",
    "for filter_arn in interaction_filter_arns:\n",
    "    recommendations_df_events = get_new_recommendations_df_by_filter(recommendations_df_events, user, filter_arn)\n",
    "    \n",
    "recommendations_df_events"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "now lets send a watch event in for 4 unwatched recommendations, which would simulate watching 4 movies. In a VOD application, you may choose to send in an event after they have watched a significant amount (over 75%) of a piece of content. Sending at 100% complete could miss people that stop short of the credits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sending event watch for Shrek 2 (2004)\n",
      "sending event watch for Casino Royale (2006)\n",
      "sending event watch for Little Miss Sunshine (2006)\n",
      "sending event watch for Avatar (2009)\n"
     ]
    }
   ],
   "source": [
    " # Get the recommendations\n",
    "top_unwatched_recommendations_response = personalize_runtime.get_recommendations(\n",
    "    campaignArn = userpersonalization_campaign_arn,\n",
    "    userId = str(user_id),\n",
    "    filterArn = filter_arn,\n",
    "    numResults=4)\n",
    "item_list = top_unwatched_recommendations_response['itemList']\n",
    "for item in item_list:\n",
    "    print('sending event watch for ' + get_movie_by_id(item['itemId']))\n",
    "    send_movie_click(USER_ID=user_id, ITEM_ID=item['itemId'], EVENT_TYPE='watch')\n",
    "    time.sleep(10)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can look at the event filters to see the updated watched and unwatched recommendations "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>watched</th>\n",
       "      <th>unwatched</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Avatar (2009)</td>\n",
       "      <td>Sound of Music, The (1965)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Star Trek (2009)</td>\n",
       "      <td>Lady and the Tramp (1955)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>V for Vendetta (2006)</td>\n",
       "      <td>Cinderella (1950)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>300 (2007)</td>\n",
       "      <td>Little Mermaid, The (1989)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Harry Potter and the Order of the Phoenix (2007)</td>\n",
       "      <td>My Fair Lady (1964)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "      <td>It's a Wonderful Life (1946)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Batman Begins (2005)</td>\n",
       "      <td>King and I, The (1956)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Casino Royale (2006)</td>\n",
       "      <td>Meet Me in St. Louis (1944)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Lion King, The (1994)</td>\n",
       "      <td>Doctor Dolittle (1967)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Prestige, The (2006)</td>\n",
       "      <td>Shrek 2 (2004)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Dumbo (1941)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Chitty Chitty Bang Bang (1968)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Music Man, The (1962)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>NaN</td>\n",
       "      <td>American in Paris, An (1951)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Night at the Opera, A (1935)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Mulan (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>NaN</td>\n",
       "      <td>North by Northwest (1959)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Sword in the Stone, The (1963)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Peter Pan (1953)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>NaN</td>\n",
       "      <td>West Side Story (1961)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Beauty and the Beast (1991)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Alice in Wonderland (1951)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Brigadoon (1954)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Fantasia (1940)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Guys and Dolls (1955)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             watched  \\\n",
       "0                                      Avatar (2009)   \n",
       "1                                   Star Trek (2009)   \n",
       "2                              V for Vendetta (2006)   \n",
       "3                                         300 (2007)   \n",
       "4   Harry Potter and the Order of the Phoenix (2007)   \n",
       "5                                     Shrek 2 (2004)   \n",
       "6                               Batman Begins (2005)   \n",
       "7                               Casino Royale (2006)   \n",
       "8                              Lion King, The (1994)   \n",
       "9                               Prestige, The (2006)   \n",
       "10                                               NaN   \n",
       "11                                               NaN   \n",
       "12                                               NaN   \n",
       "13                                               NaN   \n",
       "14                                               NaN   \n",
       "15                                               NaN   \n",
       "16                                               NaN   \n",
       "17                                               NaN   \n",
       "18                                               NaN   \n",
       "19                                               NaN   \n",
       "20                                               NaN   \n",
       "21                                               NaN   \n",
       "22                                               NaN   \n",
       "23                                               NaN   \n",
       "24                                               NaN   \n",
       "\n",
       "                         unwatched  \n",
       "0       Sound of Music, The (1965)  \n",
       "1        Lady and the Tramp (1955)  \n",
       "2                Cinderella (1950)  \n",
       "3       Little Mermaid, The (1989)  \n",
       "4              My Fair Lady (1964)  \n",
       "5     It's a Wonderful Life (1946)  \n",
       "6           King and I, The (1956)  \n",
       "7      Meet Me in St. Louis (1944)  \n",
       "8           Doctor Dolittle (1967)  \n",
       "9                   Shrek 2 (2004)  \n",
       "10                    Dumbo (1941)  \n",
       "11  Chitty Chitty Bang Bang (1968)  \n",
       "12           Music Man, The (1962)  \n",
       "13    American in Paris, An (1951)  \n",
       "14    Night at the Opera, A (1935)  \n",
       "15                    Mulan (1998)  \n",
       "16       North by Northwest (1959)  \n",
       "17  Sword in the Stone, The (1963)  \n",
       "18                Peter Pan (1953)  \n",
       "19          West Side Story (1961)  \n",
       "20     Beauty and the Beast (1991)  \n",
       "21      Alice in Wonderland (1951)  \n",
       "22                Brigadoon (1954)  \n",
       "23                 Fantasia (1940)  \n",
       "24           Guys and Dolls (1955)  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommendations_df_events = pd.DataFrame()\n",
    "for filter_arn in interaction_filter_arns:\n",
    "    recommendations_df_events = get_new_recommendations_df_by_filter(recommendations_df_events, user, filter_arn)\n",
    "    \n",
    "recommendations_df_events"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Personalized Ranking\n",
    "\n",
    "The core use case for personalized ranking is to take a collection of items and to render them in priority or probable order of interest for a user. For a VOD application you want dynamically render a personalized shelf/rail/carousel based on some information (director, location, superhero franchise, movie time period etc). This may not be information that you have in your metadata, so a item metadata filter will not work, howeverr you may have this information within you system to generate the item list. \n",
    "\n",
    "To demonstrate this, we will use the same user from before and a random collection of items."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "rerank_user = user\n",
    "rerank_items = items_df.sample(25).index.tolist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now build a nice dataframe that shows the input data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Un-Ranked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Idle Hands (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Heartless (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Wadjda (2012)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Play Misty for Me (1971)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Jazz Singer, The (1927)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Great Expectations (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Last Samurai, The (2003)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Trading Places (1983)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Porky's II: The Next Day (1983)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Shaun the Sheep Movie (2015)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Romy and Michele's High School Reunion (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Toys (1992)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Nuremberg (2000)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Bride Wars (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Mask, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Welcome to Happiness (2015)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Dog Park (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Golden Child, The (1986)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Six Days Seven Nights (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Pacific Heights (1990)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Man Apart, A (2003)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Diamond Arm, The (Brilliantovaya ruka) (1968)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>My Favorite Martian (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Mobsters (1991)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Entrapment (1999)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                        Un-Ranked\n",
       "0                               Idle Hands (1999)\n",
       "1                                Heartless (2009)\n",
       "2                                   Wadjda (2012)\n",
       "3                        Play Misty for Me (1971)\n",
       "4                         Jazz Singer, The (1927)\n",
       "5                       Great Expectations (1998)\n",
       "6                        Last Samurai, The (2003)\n",
       "7                           Trading Places (1983)\n",
       "8                 Porky's II: The Next Day (1983)\n",
       "9                    Shaun the Sheep Movie (2015)\n",
       "10  Romy and Michele's High School Reunion (1997)\n",
       "11                                    Toys (1992)\n",
       "12                               Nuremberg (2000)\n",
       "13                              Bride Wars (2009)\n",
       "14                               Mask, The (1994)\n",
       "15                    Welcome to Happiness (2015)\n",
       "16                                Dog Park (1998)\n",
       "17                       Golden Child, The (1986)\n",
       "18                   Six Days Seven Nights (1998)\n",
       "19                         Pacific Heights (1990)\n",
       "20                            Man Apart, A (2003)\n",
       "21  Diamond Arm, The (Brilliantovaya ruka) (1968)\n",
       "22                     My Favorite Martian (1999)\n",
       "23                                Mobsters (1991)\n",
       "24                              Entrapment (1999)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rerank_list = []\n",
    "for item in rerank_items:\n",
    "    movie = get_movie_by_id(item)\n",
    "    rerank_list.append(movie)\n",
    "rerank_df = pd.DataFrame(rerank_list, columns = ['Un-Ranked'])\n",
    "rerank_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then make the personalized ranking API call."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ResponseMetadata': {'RequestId': 'f1f41c69-7541-45e8-a31e-f92e4c799bd5',\n",
       "  'HTTPStatusCode': 200,\n",
       "  'HTTPHeaders': {'content-type': 'application/json',\n",
       "   'date': 'Tue, 15 Sep 2020 03:33:12 GMT',\n",
       "   'x-amzn-requestid': 'f1f41c69-7541-45e8-a31e-f92e4c799bd5',\n",
       "   'content-length': '1436',\n",
       "   'connection': 'keep-alive'},\n",
       "  'RetryAttempts': 0},\n",
       " 'personalizedRanking': [{'itemId': '7143', 'score': 0.9174433},\n",
       "  {'itemId': '2606', 'score': 0.0124089},\n",
       "  {'itemId': '65585', 'score': 0.0094577},\n",
       "  {'itemId': '1894', 'score': 0.0076884},\n",
       "  {'itemId': '1513', 'score': 0.0066342},\n",
       "  {'itemId': '66544', 'score': 0.0065695},\n",
       "  {'itemId': '101070', 'score': 0.0062184},\n",
       "  {'itemId': '6860', 'score': 0.0053613},\n",
       "  {'itemId': '2605', 'score': 0.005054},\n",
       "  {'itemId': '2498', 'score': 0.0050442},\n",
       "  {'itemId': '3039', 'score': 0.0037912},\n",
       "  {'itemId': '131656', 'score': 0.0035436},\n",
       "  {'itemId': '2253', 'score': 0.0027009},\n",
       "  {'itemId': '1735', 'score': 0.0023023},\n",
       "  {'itemId': '2735', 'score': 0.0019111},\n",
       "  {'itemId': '151653', 'score': 0.001728},\n",
       "  {'itemId': '3219', 'score': 0.000998},\n",
       "  {'itemId': '4803', 'score': 0.0007123},\n",
       "  {'itemId': '367', 'score': 0.0004327},\n",
       "  {'itemId': '79868'},\n",
       "  {'itemId': '25757'},\n",
       "  {'itemId': '3689'},\n",
       "  {'itemId': '2884'},\n",
       "  {'itemId': '6280'},\n",
       "  {'itemId': '26184'}],\n",
       " 'recommendationId': 'RID-505bd17a-5cb0-4a7a-84e8-638da186e482'}"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Convert user to string:\n",
    "user_id = str(rerank_user)\n",
    "rerank_item_list = []\n",
    "for item in rerank_items:\n",
    "    rerank_item_list.append(str(item))\n",
    "    \n",
    "# Get recommended reranking\n",
    "get_recommendations_response_rerank = personalize_runtime.get_personalized_ranking(\n",
    "        campaignArn = rerank_campaign_arn,\n",
    "        userId = user_id,\n",
    "        inputList = rerank_item_list\n",
    ")\n",
    "\n",
    "get_recommendations_response_rerank"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now add the reranked items as a second column to the original dataframe, for a side-by-side comparison."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Un-Ranked</th>\n",
       "      <th>Re-Ranked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Idle Hands (1999)</td>\n",
       "      <td>Last Samurai, The (2003)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Heartless (2009)</td>\n",
       "      <td>Idle Hands (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Wadjda (2012)</td>\n",
       "      <td>Bride Wars (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Play Misty for Me (1971)</td>\n",
       "      <td>Six Days Seven Nights (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Jazz Singer, The (1927)</td>\n",
       "      <td>Romy and Michele's High School Reunion (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Great Expectations (1998)</td>\n",
       "      <td>Nuremberg (2000)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Last Samurai, The (2003)</td>\n",
       "      <td>Wadjda (2012)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Trading Places (1983)</td>\n",
       "      <td>Mobsters (1991)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Porky's II: The Next Day (1983)</td>\n",
       "      <td>Entrapment (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Shaun the Sheep Movie (2015)</td>\n",
       "      <td>My Favorite Martian (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Romy and Michele's High School Reunion (1997)</td>\n",
       "      <td>Trading Places (1983)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Toys (1992)</td>\n",
       "      <td>Shaun the Sheep Movie (2015)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Nuremberg (2000)</td>\n",
       "      <td>Toys (1992)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Bride Wars (2009)</td>\n",
       "      <td>Great Expectations (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Mask, The (1994)</td>\n",
       "      <td>Golden Child, The (1986)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Welcome to Happiness (2015)</td>\n",
       "      <td>Welcome to Happiness (2015)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Dog Park (1998)</td>\n",
       "      <td>Pacific Heights (1990)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Golden Child, The (1986)</td>\n",
       "      <td>Play Misty for Me (1971)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Six Days Seven Nights (1998)</td>\n",
       "      <td>Mask, The (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Pacific Heights (1990)</td>\n",
       "      <td>Heartless (2009)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Man Apart, A (2003)</td>\n",
       "      <td>Jazz Singer, The (1927)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Diamond Arm, The (Brilliantovaya ruka) (1968)</td>\n",
       "      <td>Porky's II: The Next Day (1983)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>My Favorite Martian (1999)</td>\n",
       "      <td>Dog Park (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Mobsters (1991)</td>\n",
       "      <td>Man Apart, A (2003)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Entrapment (1999)</td>\n",
       "      <td>Diamond Arm, The (Brilliantovaya ruka) (1968)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                        Un-Ranked  \\\n",
       "0                               Idle Hands (1999)   \n",
       "1                                Heartless (2009)   \n",
       "2                                   Wadjda (2012)   \n",
       "3                        Play Misty for Me (1971)   \n",
       "4                         Jazz Singer, The (1927)   \n",
       "5                       Great Expectations (1998)   \n",
       "6                        Last Samurai, The (2003)   \n",
       "7                           Trading Places (1983)   \n",
       "8                 Porky's II: The Next Day (1983)   \n",
       "9                    Shaun the Sheep Movie (2015)   \n",
       "10  Romy and Michele's High School Reunion (1997)   \n",
       "11                                    Toys (1992)   \n",
       "12                               Nuremberg (2000)   \n",
       "13                              Bride Wars (2009)   \n",
       "14                               Mask, The (1994)   \n",
       "15                    Welcome to Happiness (2015)   \n",
       "16                                Dog Park (1998)   \n",
       "17                       Golden Child, The (1986)   \n",
       "18                   Six Days Seven Nights (1998)   \n",
       "19                         Pacific Heights (1990)   \n",
       "20                            Man Apart, A (2003)   \n",
       "21  Diamond Arm, The (Brilliantovaya ruka) (1968)   \n",
       "22                     My Favorite Martian (1999)   \n",
       "23                                Mobsters (1991)   \n",
       "24                              Entrapment (1999)   \n",
       "\n",
       "                                        Re-Ranked  \n",
       "0                        Last Samurai, The (2003)  \n",
       "1                               Idle Hands (1999)  \n",
       "2                               Bride Wars (2009)  \n",
       "3                    Six Days Seven Nights (1998)  \n",
       "4   Romy and Michele's High School Reunion (1997)  \n",
       "5                                Nuremberg (2000)  \n",
       "6                                   Wadjda (2012)  \n",
       "7                                 Mobsters (1991)  \n",
       "8                               Entrapment (1999)  \n",
       "9                      My Favorite Martian (1999)  \n",
       "10                          Trading Places (1983)  \n",
       "11                   Shaun the Sheep Movie (2015)  \n",
       "12                                    Toys (1992)  \n",
       "13                      Great Expectations (1998)  \n",
       "14                       Golden Child, The (1986)  \n",
       "15                    Welcome to Happiness (2015)  \n",
       "16                         Pacific Heights (1990)  \n",
       "17                       Play Misty for Me (1971)  \n",
       "18                               Mask, The (1994)  \n",
       "19                               Heartless (2009)  \n",
       "20                        Jazz Singer, The (1927)  \n",
       "21                Porky's II: The Next Day (1983)  \n",
       "22                                Dog Park (1998)  \n",
       "23                            Man Apart, A (2003)  \n",
       "24  Diamond Arm, The (Brilliantovaya ruka) (1968)  "
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ranked_list = []\n",
    "item_list = get_recommendations_response_rerank['personalizedRanking']\n",
    "for item in item_list:\n",
    "    movie = get_movie_by_id(item['itemId'])\n",
    "    ranked_list.append(movie)\n",
    "ranked_df = pd.DataFrame(ranked_list, columns = ['Re-Ranked'])\n",
    "rerank_df = pd.concat([rerank_df, ranked_df], axis=1)\n",
    "rerank_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see above how each entry was re-ordered based on the model's understanding of the user. This is a popular task when you have a collection of items to surface a user, a list of promotions for example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Batch recommendations <a class=\"anchor\" id=\"batch\"></a>\n",
    "[Back to top](#top)\n",
    "\n",
    "There are many cases where you may want to have a larger dataset of exported recommendations. Recently, Amazon Personalize launched batch recommendations as a way to export a collection of recommendations to S3. In this example, we will walk through how to do this for the HRNN solution. For more information about batch recommendations, please see the [documentation](https://docs.aws.amazon.com/personalize/latest/dg/getting-recommendations.html#recommendations-batch). This feature applies to all recipes, but the output format will vary.\n",
    "\n",
    "A simple implementation looks like this:\n",
    "\n",
    "```python\n",
    "import boto3\n",
    "\n",
    "personalize_rec = boto3.client(service_name='personalize')\n",
    "\n",
    "personalize_rec.create_batch_inference_job (\n",
    "    solutionVersionArn = \"Solution version ARN\",\n",
    "    jobName = \"Batch job name\",\n",
    "    roleArn = \"IAM role ARN\",\n",
    "    jobInput = \n",
    "       {\"s3DataSource\": {\"path\": S3 input path}},\n",
    "    jobOutput = \n",
    "       {\"s3DataDestination\": {\"path\":S3 output path\"}}\n",
    ")\n",
    "```\n",
    "\n",
    "The SDK import, the solution version arn, and role arns have all been determined. This just leaves an input, an output, and a job name to be defined.\n",
    "\n",
    "Starting with the input for HRNN, it looks like:\n",
    "\n",
    "\n",
    "```JSON\n",
    "{\"userId\": \"4638\"}\n",
    "{\"userId\": \"663\"}\n",
    "{\"userId\": \"3384\"}\n",
    "```\n",
    "\n",
    "This should yield an output that looks like this:\n",
    "\n",
    "```JSON\n",
    "{\"input\":{\"userId\":\"4638\"}, \"output\": {\"recommendedItems\": [\"296\", \"1\", \"260\", \"318\"]}}\n",
    "{\"input\":{\"userId\":\"663\"}, \"output\": {\"recommendedItems\": [\"1393\", \"3793\", \"2701\", \"3826\"]}}\n",
    "{\"input\":{\"userId\":\"3384\"}, \"output\": {\"recommendedItems\": [\"8368\", \"5989\", \"40815\", \"48780\"]}}\n",
    "```\n",
    "\n",
    "The output is a JSON Lines file. It consists of individual JSON objects, one per line. So we will need to put in more work later to digest the results in this format."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Building the input file\n",
    "\n",
    "When you are using the batch feature, you specify the users that you'd like to receive recommendations for when the job has completed. The cell below will again select a few random users and will then build the file and save it to disk. From there, you will upload it to S3 to use in the API call later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We will use the same users from before\n",
    "users\n",
    "# Write the file to disk\n",
    "json_input_filename = \"json_input.json\"\n",
    "with open(data_dir + \"/\" + json_input_filename, 'w') as json_input:\n",
    "    for user_id in users:\n",
    "        json_input.write('{\"userId\": \"' + str(user_id) + '\"}\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"userId\": \"346\"}\r\n",
      "{\"userId\": \"75\"}\r\n",
      "{\"userId\": \"247\"}\r\n"
     ]
    }
   ],
   "source": [
    "# Showcase the input file:\n",
    "!cat $data_dir\"/\"$json_input_filename"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Upload the file to S3 and save the path as a variable for later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "s3://835319576252personalizepocvod/json_input.json\n"
     ]
    }
   ],
   "source": [
    "# Upload files to S3\n",
    "boto3.Session().resource('s3').Bucket(bucket_name).Object(json_input_filename).upload_file(data_dir+\"/\"+json_input_filename)\n",
    "s3_input_path = \"s3://\" + bucket_name + \"/\" + json_input_filename\n",
    "print(s3_input_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Batch recommendations read the input from the file we've uploaded to S3. Similarly, batch recommendations will save the output to file in S3. So we define the output path where the results should be saved."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "s3://835319576252personalizepocvod/\n"
     ]
    }
   ],
   "source": [
    "# Define the output path\n",
    "s3_output_path = \"s3://\" + bucket_name + \"/\"\n",
    "print(s3_output_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now just make the call to kick off the batch export process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "batchInferenceJobArn = personalize.create_batch_inference_job (\n",
    "    solutionVersionArn = userpersonalization_solution_version_arn,\n",
    "    jobName = \"VOD-POC-Batch-Inference-Job-UserPersonalization_\" + str(round(time.time()*1000)),\n",
    "    roleArn = role_arn,\n",
    "    jobInput = \n",
    "     {\"s3DataSource\": {\"path\": s3_input_path}},\n",
    "    jobOutput = \n",
    "     {\"s3DataDestination\":{\"path\": s3_output_path}}\n",
    ")\n",
    "batchInferenceJobArn = batchInferenceJobArn['batchInferenceJobArn']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the while loop below to track the status of the batch recommendation call. This can take around 30 minutes to complete, because Personalize needs to stand up the infrastructure to perform the task. We are testing the feature with a dataset of only 3 users, which is not an efficient use of this mechanism. Normally, you would only use this feature for bulk processing, in which case the efficiencies will become clear."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Import Started on:  04:01:17 AM\n",
      "DatasetInferenceJob: ACTIVE\n",
      "Import Completed on:  04:01:17 AM\n",
      "CPU times: user 4.37 ms, sys: 114 µs, total: 4.48 ms\n",
      "Wall time: 29.9 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "current_time = datetime.now()\n",
    "print(\"Import Started on: \", current_time.strftime(\"%I:%M:%S %p\"))\n",
    "\n",
    "max_time = time.time() + 6*60*60 # 6 hours\n",
    "while time.time() < max_time:\n",
    "    describe_dataset_inference_job_response = personalize.describe_batch_inference_job(\n",
    "        batchInferenceJobArn = batchInferenceJobArn\n",
    "    )\n",
    "    status = describe_dataset_inference_job_response[\"batchInferenceJob\"]['status']\n",
    "    print(\"DatasetInferenceJob: {}\".format(status))\n",
    "    \n",
    "    if status == \"ACTIVE\" or status == \"CREATE FAILED\":\n",
    "        break\n",
    "        \n",
    "    time.sleep(60)\n",
    "    \n",
    "current_time = datetime.now()\n",
    "print(\"Import Completed on: \", current_time.strftime(\"%I:%M:%S %p\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "s3 = boto3.client('s3')\n",
    "export_name = json_input_filename + \".out\"\n",
    "s3.download_file(bucket_name, export_name, data_dir+\"/\"+export_name)\n",
    "\n",
    "# Update DF rendering\n",
    "pd.set_option('display.max_rows', 30)\n",
    "with open(data_dir+\"/\"+export_name) as json_file:\n",
    "    # Get the first line and parse it\n",
    "    line = json.loads(json_file.readline())\n",
    "    # Do the same for the other lines\n",
    "    while line:\n",
    "        # extract the user ID \n",
    "        col_header = \"User: \" + line['input']['userId']\n",
    "        # Create a list for all the artists\n",
    "        recommendation_list = []\n",
    "        # Add all the entries\n",
    "        for item in line['output']['recommendedItems']:\n",
    "            movie = get_movie_by_id(item)\n",
    "            recommendation_list.append(movie)\n",
    "        if 'bulk_recommendations_df' in locals():\n",
    "            new_rec_DF = pd.DataFrame(recommendation_list, columns = [col_header])\n",
    "            bulk_recommendations_df = bulk_recommendations_df.join(new_rec_DF)\n",
    "        else:\n",
    "            bulk_recommendations_df = pd.DataFrame(recommendation_list, columns=[col_header])\n",
    "        try:\n",
    "            line = json.loads(json_file.readline())\n",
    "        except:\n",
    "            line = None\n",
    "bulk_recommendations_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Wrap up <a class=\"anchor\" id=\"wrapup\"></a>\n",
    "[Back to top](#top)\n",
    "\n",
    "With that you now have a fully working collection of models to tackle various recommendation and personalization scenarios, as well as the skills to manipulate customer data to better integrate with the service, and a knowledge of how to do all this over APIs and by leveraging open source data science tools.\n",
    "\n",
    "Use these notebooks as a guide to getting started with your customers for POCs. As you find missing components, or discover new approaches, cut a pull request and provide any additional helpful components that may be missing from this collection.\n",
    "\n",
    "You'll want to make sure that you clean up all of the resources deployed during this POC. We have provided a separate notebook which shows you how to identify and delete the resources in `04_Clean_Up_Resources.ipynb`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%store event_tracker_arn\n",
    "%store batchInferenceJobArn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_python3",
   "language": "python",
   "name": "conda_python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
